-- Logs begin at Sun 2022-09-11 07:53:43 UTC, end at Sun 2022-09-11 09:08:33 UTC. -- Sep 11 08:24:08.015198 dv-cinder-856941-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 11 08:24:08.027032 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 11 08:24:08.029107 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 11 08:24:08 2022] *** Sep 11 08:24:08.029107 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 11 08:24:08.029107 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 11 08:24:08.029107 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: nodename: dv-cinder-856941-2-iscsi Sep 11 08:24:08.029107 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: machine: x86_64 Sep 11 08:24:08.029107 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: clock source: unix Sep 11 08:24:08.029107 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: pcre jit disabled Sep 11 08:24:08.029107 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: detected number of CPU cores: 4 Sep 11 08:24:08.029107 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: current working directory: / Sep 11 08:24:08.029107 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: detected binary path: /usr/bin/uwsgi-core Sep 11 08:24:08.029776 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: your processes number limit is 47680 Sep 11 08:24:08.029776 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: your memory page size is 4096 bytes Sep 11 08:24:08.029776 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: detected max file descriptor number: 2048 Sep 11 08:24:08.029776 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: lock engine: pthread robust mutexes Sep 11 08:24:08.029776 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: thunder lock: enabled Sep 11 08:24:08.029776 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 11 08:24:08.029776 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 11 08:24:08.070265 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: Python main interpreter initialized at 0x560affbcd9c0 Sep 11 08:24:08.070265 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: python threads support enabled Sep 11 08:24:08.070265 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: your server socket listen backlog is limited to 100 connections Sep 11 08:24:08.070265 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: your mercy for graceful operations on workers is 90 seconds Sep 11 08:24:08.070728 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: mapped 403077 bytes (393 KB) for 2 cores Sep 11 08:24:08.070728 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: *** Operational MODE: preforking *** Sep 11 08:24:08.070985 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: *** uWSGI is running in multiple interpreter mode *** Sep 11 08:24:08.070985 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: spawned uWSGI master process (pid: 81559) Sep 11 08:24:08.071209 dv-cinder-856941-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 11 08:24:08.075455 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: spawned uWSGI worker 1 (pid: 81570, cores: 1) Sep 11 08:24:08.075455 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: spawned uWSGI worker 2 (pid: 81571, cores: 1) Sep 11 08:24:08.075455 dv-cinder-856941-2-iscsi devstack@keystone.service[81559]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 11 08:24:12.469305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 08:24:12.469305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: from cryptography.utils import int_from_bytes Sep 11 08:24:12.470178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 08:24:12.470178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: from cryptography.utils import int_from_bytes Sep 11 08:24:12.476562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 08:24:12.476562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: from cryptography.utils import int_from_bytes Sep 11 08:24:12.477226 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 08:24:12.477226 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: from cryptography.utils import int_from_bytes Sep 11 08:24:12.521935 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 08:24:12.522267 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 08:24:12.522384 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] command line args: [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 08:24:12.522581 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 08:24:12.522759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ================================================================================ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 08:24:12.523015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] admin_token = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.523265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config_dir = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.523457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.523637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config_source = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.523818 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] debug = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.524020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.525214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.525214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.525214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.525214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.525469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.525469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_config_append = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.525576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.525747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_dir = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.525926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.526177 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.526429 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.526677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.526925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.527103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.527280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.527523 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.527768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.528002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.528235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.528389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 08:24:12.528476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.528603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 08:24:12.528679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.528801 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] command line args: [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 08:24:12.528921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.529037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 08:24:12.529150 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.529234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ================================================================================ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 08:24:12.529427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.529512 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] admin_token = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.529689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_dir = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.529878 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.529955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.530079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_source = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.530186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] publish_errors = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.530281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] debug = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.530375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.530453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.530548 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.530677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.530799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.530873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.530943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.531175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.531345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.531425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] standard_threads = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.531577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.531655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.531757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_config_append = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.531841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.531922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.531999 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.532104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_dir = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.532193 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_journal = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.532284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.532397 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_json = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.532487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_stderr = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.532564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.532640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_syslog = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.532755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.532871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.533004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.533136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.533261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.533361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.533446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.533579 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.533662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.533824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.533920 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.534077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.534176 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.534339 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.534419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.534570 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.534679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.535383 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.535466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.535620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.535701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.535870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.535957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.536119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.536197 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.536398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.536478 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.536648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.536888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.536966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.537109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] publish_errors = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.537187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.537290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.537378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.537524 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.537614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.537686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.537834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.537939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.538089 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.538196 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.538411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.538502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] standard_threads = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.538602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.538681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.538855 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.538940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.539043 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.539123 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.539223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_journal = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.539358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.539438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_json = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.539564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_stderr = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.539650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.539724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_syslog = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.539828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.539959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 08:24:12.540047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.540221 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.540301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.540463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.540548 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.540728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.540807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.540972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.541057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.541199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.541284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.541430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.541510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.541680 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.541761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.541921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.542000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.542180 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.542292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.542448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.542516 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.542649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.542725 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.542887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.542973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.543124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.543203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.543354 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.543431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.543583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.543667 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.543814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.543893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.544048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.544137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.544284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.544381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.544516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.544594 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.544748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.544827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.544984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.545064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.545215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.545300 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.545450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.545536 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.545684 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.545795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.545932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.546038 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.546209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.546319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.546457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.546564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.546669 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.546766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.546884 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.546966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.547120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.547211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.547355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.547437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.547584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.547669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.547819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.547908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.548069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.548148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.548320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.548403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.548559 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.548641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.548811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.548902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.549049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.549128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.549289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.549412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.549576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.549674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.549749 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.549809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.549979 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.550072 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.550277 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.550371 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.550496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.550584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.550736 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.550816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.550987 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.551078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.551221 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.551300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.551462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.551581 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.551694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.551783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.551932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.552011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.552168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.552249 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.552414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.552505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.552645 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.552723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.552883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.552963 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.553119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.553242 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.553355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.553435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.553588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.553673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.553822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.553908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.554056 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.554134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.554356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.554449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.554570 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.554838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.554838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.554977 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.555049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.555129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.555301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.555454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.555600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.555727 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.555854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.555981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.555981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.556220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.556354 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.556480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.556603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.556728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.556852 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.556978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.557123 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.557246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.557373 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.557498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.557625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.557749 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.557881 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.558006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.558131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.558327 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.558454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.558581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.558716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.558856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.558980 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.559106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.559231 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.559355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.559486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.559612 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.559737 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.external = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.559868 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.559994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.560119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.560251 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.560377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.560504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.560640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.560766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.password = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.560891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.561023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.token = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.561187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.561266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.561373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.561505 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.561638 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.561773 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.561901 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.562027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.562167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.562291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.562424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.562562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.562690 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.562816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.562950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.563101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.563237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.563365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.563492 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.563625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.563753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.563879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.564008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.564144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.564271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.564405 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.564496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.564659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.564786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.564919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.565047 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.565184 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.565310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.565436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.565518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.565580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.565640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.565640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.565755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.565822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.565884 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.565944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.566008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.566075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.external = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.566136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.566375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.566471 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.566471 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.566605 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.566689 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.566763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.566886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.566963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.password = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.566963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.token = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.567092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.567092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.567233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.567233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.567370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.567370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.567570 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.567570 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.567702 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.567702 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.567838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.567967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.568092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.568170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.568249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.568392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.568473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.568634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.568726 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.568882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.568960 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.569128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.569207 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.569365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.569452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.569600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.569680 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.569837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.569915 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.570067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.570264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.570361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.570437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.570565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.570646 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.570815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.570903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.571047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.571128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.571280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.571360 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.571515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.571602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.571759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.571839 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.572004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.572083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.572250 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.572338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.572504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.572582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.572746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.572831 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.573004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.573173 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.573256 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.573330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.573455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.573536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.573683 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.573774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.573920 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.574000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.574167 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.574344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.574477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.574622 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.574705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.574778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.574857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.574938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.575089 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.575179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.575321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.575402 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.575568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.575649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.575809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.575896 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.576043 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.576122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.576289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.576369 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.576525 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.576614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.576756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.576835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.576992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.577116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.577260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.577368 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.577486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.577567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.577730 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.577814 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.577897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.577971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.578116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.578270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.578401 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.578519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.578654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.578772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.578912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.579023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.579161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.579242 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.579436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.579518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.579678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.579758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.579925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.580057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.580174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.580252 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.580384 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.580471 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.580617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.580698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.580846 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.580926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.581086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.581169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.581324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.581403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.582096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.582236 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.582357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.582436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.582604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.582692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.582849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.582930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.583083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.583162 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.583312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.583392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.583549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.583631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.583779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.583859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.584012 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.584092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.584240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.584320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.584487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.584576 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.584722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.584919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.585011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.585084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.585190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.585269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.585428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.585508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.585656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.585737 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.585902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.585983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.586134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.586279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.586392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.586474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.586624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.586703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.586859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.586947 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.587102 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.587187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.587338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.587417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.587571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.587705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.587824 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.587943 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.588056 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.588430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.588554 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.588701 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.588790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.588935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.589015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.589186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.589266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.589436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.589523 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.589687 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.589767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.589916 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.589997 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.590215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.590314 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.590390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.590462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.590631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.590711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.590870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.591027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.591103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.591205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.591341 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.591420 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.591588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.591668 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.591826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.591905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.592071 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.592150 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.592305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.592385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.592538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.592617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.592772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.592851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.593010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.593090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.593239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.593319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.593474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.593553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.593712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.593792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.593957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.594037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.594228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.594310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.594475 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.594557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.594719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.594799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.594953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.595031 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.595201 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.595281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.595434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.595515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.595670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.595747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.595909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.595990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.596154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.596233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.596397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.596476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.596631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.596829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.596948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.597051 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.597225 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.597850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.597948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.598107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.598266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.598370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.598598 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.598680 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.598837 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.598918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.599091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.599171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.599327 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.599406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.599566 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.599646 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.599800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.599880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.600031 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.600112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.600263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.600344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.600508 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.600587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.600744 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.600824 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.600994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.601075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.601234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.601314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.601475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.601739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.601854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.601932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.602021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.602134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.602265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.602360 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.602459 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.602540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.602711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.602793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.602947 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.603037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.603198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.603278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.603433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.603511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.603675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.603763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.603918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.603998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.604163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.604347 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.604430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.604587 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.604669 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.604839 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.604920 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.605081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.605160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.605330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.605410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.605575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.605662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.605808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.605889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.606065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.606212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.606346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.606426 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.606585 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.606664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.606821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.606914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.607056 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.607203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.607281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.607352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.607518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.607606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.607756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.607837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.607989 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.608070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.608238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.608360 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.608442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.608503 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.608677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.608758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.608915 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.609004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.609150 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.609230 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.609394 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.609473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.609629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.609718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.609873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.609954 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.610104 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.610209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.610375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.610468 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.610610 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.610692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.610764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 08:24:12.610837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.611091 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.611334 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.611575 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.611813 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.612061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.612345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.612578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.612978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.613255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.613520 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.613769 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.614024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.614288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.614537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.614777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.615016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.615254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.615494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.615747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.615981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.616226 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.616465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.616714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.616953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.617200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.617434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.617677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.617914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 08:24:12.618088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 08:24:12.641526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.641730 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.641952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.645714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.645921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.646168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.652148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.655739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.683772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.683984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.684806 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.685044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.742183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.742438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.792066 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.792298 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.793442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.793684 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.951249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.951475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.951596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.951699 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.951787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.951918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.952008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.952135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.952224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.952349 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.952444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.952602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.976440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.976641 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.976715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.976830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.976936 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.977015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.977091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.977179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.977291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.977381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.977495 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.977596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.977673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.977777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.988993 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:12.989207 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.001785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.001936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.002054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.002201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.002285 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.002362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.002468 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.002570 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.002648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.002785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.002910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.003037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.003115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.003208 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 08:24:13.020815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.021025 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 08:24:13.021346 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.021434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.021559 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 08:24:13.021674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 08:24:13.021835 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.021955 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.022093 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 08:24:13.022310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.022419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.022540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.022621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.022729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.022898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.023168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.032245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.032526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.032797 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.032911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.033080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.033196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.033358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.033649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.040644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.041223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.043582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.043865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.044155 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.044240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.044443 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.044527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.044731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.044816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.044996 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.045080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.045273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.045358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.045549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.045634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.045825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.045908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.046100 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.046225 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.046398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.046483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.046674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.046950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.069548 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.070203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.071052 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.071344 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.071707 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.071993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.080587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.080587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.080587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.080883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.080883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.080883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.083015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.083314 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.083696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.084000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.089459 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.089755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.090043 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.090237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.090335 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.090543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.090650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.090845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.090986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.091147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.091282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.091415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.091501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.091696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.091971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.092252 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.107499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.107783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.108321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.108612 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.111677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.112505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.117988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.117988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.118184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.118184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.120035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.120732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.123183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.123859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.126428 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.126731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.127022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.127107 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.127311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.127404 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.127612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.127698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.127906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.127990 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.128205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.128518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.143726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.144550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.144823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.144936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.147957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.148059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.148059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.148059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.151968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.152712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.153519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.154303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.155081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.155361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.155860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.156144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.158129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.158436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.158723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.158926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.159015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.159227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.159388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.159518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.160361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.160756 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.171182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.171551 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.172010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.172130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.172212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.176421 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.176564 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.176688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.176804 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.176924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.177057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.177181 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.177181 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.177181 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.188584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.189525 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.191604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.192577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.194517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.195502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.195969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.196988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.197462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.197747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.198191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.198493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.198789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.199209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.204618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.205704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.206105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.207775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.209589 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.210001 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.210316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.210605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.210744 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.211148 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.211442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.211727 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.218797 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.219098 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.219383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.219903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.220209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.220502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.226990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.250908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.250908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.250908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.251119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 08:24:13.251197 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.251285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.251361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.251434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.251515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.251590 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.251652 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.251731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.251848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.251942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.252017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.252091 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.252171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.252247 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.252320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.252392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.252465 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.252617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.252694 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.252767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.252767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.252767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.252767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 08:24:13.264619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560affbcd9c0 pid: 81571 (default app) Sep 11 08:24:13.266113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560affbcd9c0 pid: 81570 (default app) Sep 11 08:24:13.268115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0223c-9242-47d7-98c5-5bcbddbdf800 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:13.268403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0223c-9242-47d7-98c5-5bcbddbdf800 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:13.268676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0223c-9242-47d7-98c5-5bcbddbdf800 None None] PATH_INFO: `/v3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:13.273364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1/1] 10.222.0.44 () {50 vars in 791 bytes} [Sun Sep 11 08:24:13 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:21.054603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb480342-156c-4c48-a3ea-14a291a2b225 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:21.055124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb480342-156c-4c48-a3ea-14a291a2b225 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:21.055124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb480342-156c-4c48-a3ea-14a291a2b225 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:21.059393 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1/2] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:24:21.064967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c89caa-ddc2-4d9b-9078-fcaa404b1892 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:21.065493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c89caa-ddc2-4d9b-9078-fcaa404b1892 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:21.065493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c89caa-ddc2-4d9b-9078-fcaa404b1892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:21.137021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: INFO dbcounter [None req-23c89caa-ddc2-4d9b-9078-fcaa404b1892 None None] Registered counter for database keystone Sep 11 08:24:21.159262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG oslo_db.sqlalchemy.engines [None req-23c89caa-ddc2-4d9b-9078-fcaa404b1892 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81571) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 08:24:21.165913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writer thread running {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 08:24:21.348477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG passlib.handlers.bcrypt [None req-23c89caa-ddc2-4d9b-9078-fcaa404b1892 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81571) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 11 08:24:21.348853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG passlib.handlers.bcrypt [None req-23c89caa-ddc2-4d9b-9078-fcaa404b1892 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81571) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 11 08:24:21.407876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-23c89caa-ddc2-4d9b-9078-fcaa404b1892 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:21.409422 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-23c89caa-ddc2-4d9b-9078-fcaa404b1892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:21.539219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 2/3] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 11 08:24:22.073951 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb480342-156c-4c48-a3ea-14a291a2b225 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:22.145456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: INFO dbcounter [None req-fb480342-156c-4c48-a3ea-14a291a2b225 None None] Registered counter for database keystone Sep 11 08:24:22.165493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG oslo_db.sqlalchemy.engines [None req-fb480342-156c-4c48-a3ea-14a291a2b225 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81570) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 08:24:22.172447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writer thread running {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 08:24:22.291245 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb480342-156c-4c48-a3ea-14a291a2b225 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:22.295489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70834843-9442-46f3-a8e2-d4b9e600b1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:22.296088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70834843-9442-46f3-a8e2-d4b9e600b1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:22.296368 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70834843-9442-46f3-a8e2-d4b9e600b1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:22.296628 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70834843-9442-46f3-a8e2-d4b9e600b1d4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:22.305366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70834843-9442-46f3-a8e2-d4b9e600b1d4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:22.358122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: WARNING keystone.common.rbac_enforcer.enforcer [None req-70834843-9442-46f3-a8e2-d4b9e600b1d4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 11 08:24:22.358487 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70834843-9442-46f3-a8e2-d4b9e600b1d4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:22.368656 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 2/4] 10.222.0.44 () {58 vars in 1324 bytes} [Sun Sep 11 08:24:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:24:22.375561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c89caa-ddc2-4d9b-9078-fcaa404b1892 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:22.393178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c89caa-ddc2-4d9b-9078-fcaa404b1892 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:22.398671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52db89c-a370-46bf-abed-742943acd864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:22.399752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52db89c-a370-46bf-abed-742943acd864 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:22.400035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52db89c-a370-46bf-abed-742943acd864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:22.400317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52db89c-a370-46bf-abed-742943acd864 admin admin] PATH_INFO: `/v3/projects/878028e7aaff4c28815b9cd862f1c36e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:22.401929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52db89c-a370-46bf-abed-742943acd864 admin admin] RBAC: Authorizing `identity:get_project(project_id=878028e7aaff4c28815b9cd862f1c36e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:22.454198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c52db89c-a370-46bf-abed-742943acd864 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 11 08:24:22.454544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52db89c-a370-46bf-abed-742943acd864 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:22.459800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 3/5] 10.222.0.44 () {56 vars in 1207 bytes} [Sun Sep 11 08:24:22 2022] GET /identity/v3/projects/878028e7aaff4c28815b9cd862f1c36e => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:24:24.647033 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcb2a0-11c7-4106-824b-d6eb9e628dc3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:24.647581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcb2a0-11c7-4106-824b-d6eb9e628dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:24.647581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcb2a0-11c7-4106-824b-d6eb9e628dc3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:24.651683 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 3/6] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:24.657433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077983c6-010d-4046-83ab-08d5b998ec6f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:24.657847 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077983c6-010d-4046-83ab-08d5b998ec6f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:24.657992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077983c6-010d-4046-83ab-08d5b998ec6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:24.678739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-077983c6-010d-4046-83ab-08d5b998ec6f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:24.680281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-077983c6-010d-4046-83ab-08d5b998ec6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:24.688056 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 4/7] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 11 08:24:24.954330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcb2a0-11c7-4106-824b-d6eb9e628dc3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:24.967091 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcb2a0-11c7-4106-824b-d6eb9e628dc3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:24.971250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763ee62-9c97-442d-b3ff-fb3e72db3b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:24.971845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763ee62-9c97-442d-b3ff-fb3e72db3b42 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:24.972120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763ee62-9c97-442d-b3ff-fb3e72db3b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:24.972389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763ee62-9c97-442d-b3ff-fb3e72db3b42 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:24.981299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763ee62-9c97-442d-b3ff-fb3e72db3b42 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:24.983154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763ee62-9c97-442d-b3ff-fb3e72db3b42 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:24.993139 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 4/8] 10.222.0.44 () {58 vars in 1324 bytes} [Sun Sep 11 08:24:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 08:24:24.999050 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077983c6-010d-4046-83ab-08d5b998ec6f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:25.008689 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077983c6-010d-4046-83ab-08d5b998ec6f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:25.013173 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b47d6-9de8-40b2-900b-0c13ee21fc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:25.013828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b47d6-9de8-40b2-900b-0c13ee21fc5e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:25.014108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b47d6-9de8-40b2-900b-0c13ee21fc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:25.014413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b47d6-9de8-40b2-900b-0c13ee21fc5e admin admin] PATH_INFO: `/v3/users/e0cb001df8d24a87b7f9253f68745c35` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:25.015761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b47d6-9de8-40b2-900b-0c13ee21fc5e admin admin] RBAC: Authorizing `identity:get_user(user_id=e0cb001df8d24a87b7f9253f68745c35)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:25.017636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b47d6-9de8-40b2-900b-0c13ee21fc5e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:25.019280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 5/9] 10.222.0.44 () {56 vars in 1198 bytes} [Sun Sep 11 08:24:24 2022] GET /identity/v3/users/e0cb001df8d24a87b7f9253f68745c35 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:27.331874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b57ba9-1015-4ebe-929f-b4a798f4920d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:27.332435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b57ba9-1015-4ebe-929f-b4a798f4920d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:27.332435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b57ba9-1015-4ebe-929f-b4a798f4920d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:27.333740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 5/10] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:27.339435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3af76-988e-4766-b8ea-7499e2b38902 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:27.339883 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3af76-988e-4766-b8ea-7499e2b38902 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:27.339971 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3af76-988e-4766-b8ea-7499e2b38902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:27.358739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-20d3af76-988e-4766-b8ea-7499e2b38902 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:27.359928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-20d3af76-988e-4766-b8ea-7499e2b38902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:27.399339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be346123-5159-4913-959a-00dc277e235e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:27.399625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be346123-5159-4913-959a-00dc277e235e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:27.400085 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be346123-5159-4913-959a-00dc277e235e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:27.401609 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 6/11] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:24:27.406833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f10114-8a34-4c62-b3a8-0654d1f6ac4e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:27.407117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f10114-8a34-4c62-b3a8-0654d1f6ac4e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:27.407396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f10114-8a34-4c62-b3a8-0654d1f6ac4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:27.444406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 6/12] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:27.451313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG passlib.handlers.bcrypt [None req-73f10114-8a34-4c62-b3a8-0654d1f6ac4e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81570) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 11 08:24:27.451711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG passlib.handlers.bcrypt [None req-73f10114-8a34-4c62-b3a8-0654d1f6ac4e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81570) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 11 08:24:27.501474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-73f10114-8a34-4c62-b3a8-0654d1f6ac4e None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:27.503136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-73f10114-8a34-4c62-b3a8-0654d1f6ac4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:27.607398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 7/13] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:24:27.710387 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3af76-988e-4766-b8ea-7499e2b38902 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:27.735741 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:27.823686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3af76-988e-4766-b8ea-7499e2b38902 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:27.865407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af96dcd-a3fe-41ff-b49c-50a3c5444f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:27.865960 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af96dcd-a3fe-41ff-b49c-50a3c5444f1b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:27.866332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af96dcd-a3fe-41ff-b49c-50a3c5444f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:27.866605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af96dcd-a3fe-41ff-b49c-50a3c5444f1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:27.900736 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f10114-8a34-4c62-b3a8-0654d1f6ac4e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:27.942588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af96dcd-a3fe-41ff-b49c-50a3c5444f1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:27.945060 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af96dcd-a3fe-41ff-b49c-50a3c5444f1b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:27.982128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:28.008398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f10114-8a34-4c62-b3a8-0654d1f6ac4e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:28.011001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:28.051766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71923093-5387-4869-bc84-664d8756ddf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:28.053740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71923093-5387-4869-bc84-664d8756ddf6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:28.054194 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71923093-5387-4869-bc84-664d8756ddf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:28.054473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71923093-5387-4869-bc84-664d8756ddf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:28.072073 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71923093-5387-4869-bc84-664d8756ddf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:24:28.076127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 7/14] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-71923093-5387-4869-bc84-664d8756ddf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:24:28.080966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:28.082030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:28.082879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:28.082879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:28.082879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:28.082879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:28.082879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:24:28.082879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:24:28.082879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:24:28.082879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:24:28.082879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:24:28.082879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:24:28.082879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:24:28.082879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:24:28.083468 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af96dcd-a3fe-41ff-b49c-50a3c5444f1b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:28.083547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 8/15] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:24:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:24:28.087721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71923093-5387-4869-bc84-664d8756ddf6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:28.197038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af96dcd-a3fe-41ff-b49c-50a3c5444f1b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:28.207443 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71923093-5387-4869-bc84-664d8756ddf6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:28.227983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:28.240616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4bcde-fd03-4b3e-b531-88d57a3bfda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:28.241734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4bcde-fd03-4b3e-b531-88d57a3bfda5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:28.241734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4bcde-fd03-4b3e-b531-88d57a3bfda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:28.241984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4bcde-fd03-4b3e-b531-88d57a3bfda5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:28.244259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6450299-4d05-432e-999a-6a04d55d089f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:28.244873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6450299-4d05-432e-999a-6a04d55d089f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:28.245150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6450299-4d05-432e-999a-6a04d55d089f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:28.245427 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6450299-4d05-432e-999a-6a04d55d089f None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:28.246104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6450299-4d05-432e-999a-6a04d55d089f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:28.246431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6450299-4d05-432e-999a-6a04d55d089f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:28.247885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6450299-4d05-432e-999a-6a04d55d089f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:28.254520 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4bcde-fd03-4b3e-b531-88d57a3bfda5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:28.256067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4bcde-fd03-4b3e-b531-88d57a3bfda5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:28.260788 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 9/16] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:24:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:24:28.266751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6450299-4d05-432e-999a-6a04d55d089f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-fce4bcde-fd03-4b3e-b531-88d57a3bfda5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:24:28.275164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:28.276139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:24:28.277096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:24:28.277096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:24:28.277096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:24:28.277096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 8/17] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:24:28.282649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4bcde-fd03-4b3e-b531-88d57a3bfda5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:28.298623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:28.379569 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6450299-4d05-432e-999a-6a04d55d089f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:28.401097 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4bcde-fd03-4b3e-b531-88d57a3bfda5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:28.418894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c4ee4-860d-489d-880a-2f8efda30f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:28.419542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c4ee4-860d-489d-880a-2f8efda30f97 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:28.419817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c4ee4-860d-489d-880a-2f8efda30f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:28.420115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c4ee4-860d-489d-880a-2f8efda30f97 None admin] PATH_INFO: `/v3/users/e0cb001df8d24a87b7f9253f68745c35` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:28.421659 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c4ee4-860d-489d-880a-2f8efda30f97 None admin] RBAC: Authorizing `identity:get_user(user_id=e0cb001df8d24a87b7f9253f68745c35)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:28.423165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c4ee4-860d-489d-880a-2f8efda30f97 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:28.424839 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 10/18] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:24:28 2022] GET /identity/v3/users/e0cb001df8d24a87b7f9253f68745c35 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:24:28.431144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c4ee4-860d-489d-880a-2f8efda30f97 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:28.441136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1990994-f944-4a75-81e9-cf8d776df221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:28.441761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1990994-f944-4a75-81e9-cf8d776df221 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:28.442041 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1990994-f944-4a75-81e9-cf8d776df221 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:28.442342 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1990994-f944-4a75-81e9-cf8d776df221 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:28.443037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1990994-f944-4a75-81e9-cf8d776df221 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:28.443341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1990994-f944-4a75-81e9-cf8d776df221 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:28.444780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1990994-f944-4a75-81e9-cf8d776df221 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:28.464058 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 9/19] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:24:28.552288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:28.560896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c4ee4-860d-489d-880a-2f8efda30f97 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:28.606691 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eabc0a-4243-4863-93bb-c7228c6e78fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:28.607385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eabc0a-4243-4863-93bb-c7228c6e78fb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:28.609110 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eabc0a-4243-4863-93bb-c7228c6e78fb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:28.609110 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eabc0a-4243-4863-93bb-c7228c6e78fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:28.614845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eabc0a-4243-4863-93bb-c7228c6e78fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:28.614845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eabc0a-4243-4863-93bb-c7228c6e78fb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:28.614845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 11/20] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:24:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:24:28.622140 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1990994-f944-4a75-81e9-cf8d776df221 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:28.664563 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:28.751852 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1990994-f944-4a75-81e9-cf8d776df221 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:28.804981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5b251-48f1-4c00-b2c2-a23684441459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:28.805610 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5b251-48f1-4c00-b2c2-a23684441459 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:28.805890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5b251-48f1-4c00-b2c2-a23684441459 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:28.806299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5b251-48f1-4c00-b2c2-a23684441459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:28.806980 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5b251-48f1-4c00-b2c2-a23684441459 None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.domain.id=default, user.id=e0cb001df8d24a87b7f9253f68745c35) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:28.807286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5b251-48f1-4c00-b2c2-a23684441459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:28.808794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5b251-48f1-4c00-b2c2-a23684441459 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:28.817685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 10/21] 10.222.0.44 () {56 vars in 1322 bytes} [Sun Sep 11 08:24:28 2022] GET /identity/v3/role_assignments?user.id=e0cb001df8d24a87b7f9253f68745c35&scope.domain.id=default&role.id=020bc02dca71485bab2312442adaa179 => generated 233 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:31.337214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45872567-5f6c-4838-89c4-9d88f2b5ab49 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:31.337761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45872567-5f6c-4838-89c4-9d88f2b5ab49 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:31.337761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45872567-5f6c-4838-89c4-9d88f2b5ab49 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:31.339668 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 12/22] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:31.345485 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a682be8-f494-46ea-950d-3b01a4fc0efd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:31.346483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a682be8-f494-46ea-950d-3b01a4fc0efd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:31.346969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a682be8-f494-46ea-950d-3b01a4fc0efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:31.372434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9a682be8-f494-46ea-950d-3b01a4fc0efd None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:31.374028 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9a682be8-f494-46ea-950d-3b01a4fc0efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:31.470445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 11/23] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:31.600396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99de983-88a9-4503-ae01-6df99b5866c2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:31.601062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99de983-88a9-4503-ae01-6df99b5866c2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:31.601552 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99de983-88a9-4503-ae01-6df99b5866c2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:31.603208 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 13/24] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:31.609211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12498fd7-6089-4f26-ba24-7241e41a3cd1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:31.609757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12498fd7-6089-4f26-ba24-7241e41a3cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:31.610317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12498fd7-6089-4f26-ba24-7241e41a3cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:31.639190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-12498fd7-6089-4f26-ba24-7241e41a3cd1 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:31.640775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-12498fd7-6089-4f26-ba24-7241e41a3cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:31.671806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:31.729896 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 12/25] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:31.915723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99de983-88a9-4503-ae01-6df99b5866c2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:32.061078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99de983-88a9-4503-ae01-6df99b5866c2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:32.076099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98038016-2d08-4b92-87a0-86ecd8b4160d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:32.076377 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98038016-2d08-4b92-87a0-86ecd8b4160d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:32.076807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98038016-2d08-4b92-87a0-86ecd8b4160d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:32.078435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 13/26] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:24:32.085246 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea900d63-b53d-4c7c-ba70-ab47edbf3118 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:32.085733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea900d63-b53d-4c7c-ba70-ab47edbf3118 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:32.086197 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea900d63-b53d-4c7c-ba70-ab47edbf3118 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:32.087838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 14/27] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:24:32.090878 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6342438-6348-4ee6-8e2d-d46d926ff657 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:32.091355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6342438-6348-4ee6-8e2d-d46d926ff657 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:32.091772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6342438-6348-4ee6-8e2d-d46d926ff657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:32.122209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998ce19-d2be-4ae3-b9af-88db60ed5010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:32.122866 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998ce19-d2be-4ae3-b9af-88db60ed5010 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:32.123144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998ce19-d2be-4ae3-b9af-88db60ed5010 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:32.123470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998ce19-d2be-4ae3-b9af-88db60ed5010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:32.124941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b6342438-6348-4ee6-8e2d-d46d926ff657 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:32.126195 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b6342438-6348-4ee6-8e2d-d46d926ff657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:32.155554 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998ce19-d2be-4ae3-b9af-88db60ed5010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:32.161219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998ce19-d2be-4ae3-b9af-88db60ed5010 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5998ce19-d2be-4ae3-b9af-88db60ed5010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:24:32.182775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:32.183708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:24:32.189631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:24:32.189631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:24:32.189631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:24:32.189631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 14/28] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:24:32.189631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd324a-061d-4e58-8b1d-f1de8a3fb640 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:32.189631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd324a-061d-4e58-8b1d-f1de8a3fb640 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:32.189631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd324a-061d-4e58-8b1d-f1de8a3fb640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:32.207142 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-13bd324a-061d-4e58-8b1d-f1de8a3fb640 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:32.209063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-13bd324a-061d-4e58-8b1d-f1de8a3fb640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:32.220698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:32.246554 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 15/29] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:32.247844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6342438-6348-4ee6-8e2d-d46d926ff657 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:32.358236 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 15/30] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:32.358236 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd324a-061d-4e58-8b1d-f1de8a3fb640 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:32.361519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:32.391352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6342438-6348-4ee6-8e2d-d46d926ff657 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:32.470072 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832ca48-649c-43a7-bb82-4ce8dbf9e605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:32.470759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832ca48-649c-43a7-bb82-4ce8dbf9e605 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:32.471201 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832ca48-649c-43a7-bb82-4ce8dbf9e605 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:32.471636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832ca48-649c-43a7-bb82-4ce8dbf9e605 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:32.500638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832ca48-649c-43a7-bb82-4ce8dbf9e605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2832ca48-649c-43a7-bb82-4ce8dbf9e605 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:24:32.509534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:32.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:32.511598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:32.511598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:32.511598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:32.511598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:32.511598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:24:32.511598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:24:32.511598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:24:32.511598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:24:32.511598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:24:32.511598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:24:32.511598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:24:32.511598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:24:32.519873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 16/31] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:24:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:24:32.533458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832ca48-649c-43a7-bb82-4ce8dbf9e605 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:32.609976 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd324a-061d-4e58-8b1d-f1de8a3fb640 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:32.681038 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3022c8f-5f57-49a4-bab6-e339a01a9d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:32.681965 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3022c8f-5f57-49a4-bab6-e339a01a9d32 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:32.684104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3022c8f-5f57-49a4-bab6-e339a01a9d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:32.684614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3022c8f-5f57-49a4-bab6-e339a01a9d32 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:32.685825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3022c8f-5f57-49a4-bab6-e339a01a9d32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:32.686390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3022c8f-5f57-49a4-bab6-e339a01a9d32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:32.694812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3022c8f-5f57-49a4-bab6-e339a01a9d32 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:32.746551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 16/32] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:32.748968 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9169367-4e73-4a2f-8bc2-0641dfb550f4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:32.749725 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9169367-4e73-4a2f-8bc2-0641dfb550f4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:32.750433 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9169367-4e73-4a2f-8bc2-0641dfb550f4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:32.753393 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 17/33] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:32.755850 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9169367-4e73-4a2f-8bc2-0641dfb550f4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:32.763979 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:32.812895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832ca48-649c-43a7-bb82-4ce8dbf9e605 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:32.876083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ab7f1-499f-439a-a4dd-9e1fa950aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:32.877130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ab7f1-499f-439a-a4dd-9e1fa950aa99 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:32.877633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ab7f1-499f-439a-a4dd-9e1fa950aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:32.878125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ab7f1-499f-439a-a4dd-9e1fa950aa99 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:32.879401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ab7f1-499f-439a-a4dd-9e1fa950aa99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:32.879945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ab7f1-499f-439a-a4dd-9e1fa950aa99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:32.883084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ab7f1-499f-439a-a4dd-9e1fa950aa99 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:32.895708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9169367-4e73-4a2f-8bc2-0641dfb550f4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:32.905859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 17/34] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:24:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:24:32.908019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d5a33-ac95-4a72-b44a-24f1281167c9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:32.908522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d5a33-ac95-4a72-b44a-24f1281167c9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:32.908988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d5a33-ac95-4a72-b44a-24f1281167c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:32.937145 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55c266-9cdd-402e-8901-eab19b93d7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:32.937673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55c266-9cdd-402e-8901-eab19b93d7f8 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:32.937945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55c266-9cdd-402e-8901-eab19b93d7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:32.938063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0a5d5a33-ac95-4a72-b44a-24f1281167c9 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:32.938261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55c266-9cdd-402e-8901-eab19b93d7f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:32.938939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55c266-9cdd-402e-8901-eab19b93d7f8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:32.940008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0a5d5a33-ac95-4a72-b44a-24f1281167c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:32.940314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55c266-9cdd-402e-8901-eab19b93d7f8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:32.945503 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:33.047278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 18/35] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:33.048331 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d5a33-ac95-4a72-b44a-24f1281167c9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:33.169160 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d5a33-ac95-4a72-b44a-24f1281167c9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:33.237550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60736ec0-09c7-4fb5-9f4a-2fc51c2febde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:33.238544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60736ec0-09c7-4fb5-9f4a-2fc51c2febde None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:33.238972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60736ec0-09c7-4fb5-9f4a-2fc51c2febde None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:33.240018 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60736ec0-09c7-4fb5-9f4a-2fc51c2febde None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:33.241060 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60736ec0-09c7-4fb5-9f4a-2fc51c2febde None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:33.243573 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60736ec0-09c7-4fb5-9f4a-2fc51c2febde None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:33.264199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:33.302771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 18/36] 10.222.0.44 () {60 vars in 1131 bytes} [Sun Sep 11 08:24:32 2022] POST /identity/v3/roles => generated 225 bytes in 548 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:24:33.305268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55c266-9cdd-402e-8901-eab19b93d7f8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:33.430303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:33.479191 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55c266-9cdd-402e-8901-eab19b93d7f8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:33.522695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4da70a-a5ae-4f62-8c40-abbead15a107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:33.523284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4da70a-a5ae-4f62-8c40-abbead15a107 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:33.523634 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4da70a-a5ae-4f62-8c40-abbead15a107 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:33.524079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4da70a-a5ae-4f62-8c40-abbead15a107 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:33.524628 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4da70a-a5ae-4f62-8c40-abbead15a107 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:33.526216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4da70a-a5ae-4f62-8c40-abbead15a107 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:33.538939 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 19/37] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:24:33 2022] POST /identity/v3/projects => generated 301 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:24:33.538939 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60736ec0-09c7-4fb5-9f4a-2fc51c2febde None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:33.655920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 19/38] 10.222.0.44 () {60 vars in 1131 bytes} [Sun Sep 11 08:24:33 2022] POST /identity/v3/roles => generated 219 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:24:33.657472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f384ae-389a-4e02-b5ee-99bdcad3f7e1 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:33.659047 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f384ae-389a-4e02-b5ee-99bdcad3f7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:33.659372 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f384ae-389a-4e02-b5ee-99bdcad3f7e1 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:33.660853 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 20/39] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:33.662477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f384ae-389a-4e02-b5ee-99bdcad3f7e1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:33.871262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60736ec0-09c7-4fb5-9f4a-2fc51c2febde None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:33.894529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f384ae-389a-4e02-b5ee-99bdcad3f7e1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:33.928653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:33.930711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270355f4-56ea-42bf-925b-bf65d1113091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:33.931534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270355f4-56ea-42bf-925b-bf65d1113091 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:33.932010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270355f4-56ea-42bf-925b-bf65d1113091 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:33.932405 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270355f4-56ea-42bf-925b-bf65d1113091 None admin] PATH_INFO: `/v3/users/e0cb001df8d24a87b7f9253f68745c35` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:33.934335 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270355f4-56ea-42bf-925b-bf65d1113091 None admin] RBAC: Authorizing `identity:get_user(user_id=e0cb001df8d24a87b7f9253f68745c35)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:33.936142 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270355f4-56ea-42bf-925b-bf65d1113091 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:33.938747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 20/40] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:24:33 2022] GET /identity/v3/users/e0cb001df8d24a87b7f9253f68745c35 => generated 247 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:33.945694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c49cd-baf8-42d5-98a9-bf28d123345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:33.945694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c49cd-baf8-42d5-98a9-bf28d123345b None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:33.945694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c49cd-baf8-42d5-98a9-bf28d123345b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:33.945694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c49cd-baf8-42d5-98a9-bf28d123345b None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:33.946473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c49cd-baf8-42d5-98a9-bf28d123345b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:33.946628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0975561-be5f-457a-bfb0-a74c80871ff7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:33.946628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0975561-be5f-457a-bfb0-a74c80871ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:33.946877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c49cd-baf8-42d5-98a9-bf28d123345b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:33.947011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0975561-be5f-457a-bfb0-a74c80871ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:33.972634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e0975561-be5f-457a-bfb0-a74c80871ff7 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:33.974418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e0975561-be5f-457a-bfb0-a74c80871ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:34.034724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 21/41] 10.222.0.44 () {60 vars in 1131 bytes} [Sun Sep 11 08:24:33 2022] POST /identity/v3/roles => generated 223 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:24:34.035783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c49cd-baf8-42d5-98a9-bf28d123345b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:34.071135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:34.089405 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 21/42] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:34.187437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c49cd-baf8-42d5-98a9-bf28d123345b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:34.231612 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3765a-a619-4c74-9b4d-d938b20e8116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:34.232269 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3765a-a619-4c74-9b4d-d938b20e8116 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:34.232548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3765a-a619-4c74-9b4d-d938b20e8116 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:34.232810 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3765a-a619-4c74-9b4d-d938b20e8116 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:34.234321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3765a-a619-4c74-9b4d-d938b20e8116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:34.235808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3765a-a619-4c74-9b4d-d938b20e8116 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:34.237828 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 22/43] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:24:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:34.245107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0975561-be5f-457a-bfb0-a74c80871ff7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:34.425419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0975561-be5f-457a-bfb0-a74c80871ff7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:34.471212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3765a-a619-4c74-9b4d-d938b20e8116 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:34.478628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2c749-88fe-43cf-af0e-c1c240455c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:34.479393 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2c749-88fe-43cf-af0e-c1c240455c5f None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:34.479733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2c749-88fe-43cf-af0e-c1c240455c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:34.480071 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2c749-88fe-43cf-af0e-c1c240455c5f None admin] PATH_INFO: `/v3/domains/default/users/e0cb001df8d24a87b7f9253f68745c35/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:34.482295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2c749-88fe-43cf-af0e-c1c240455c5f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e0cb001df8d24a87b7f9253f68745c35, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:34.484053 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2c749-88fe-43cf-af0e-c1c240455c5f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:34.539618 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:34.587218 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 22/44] 10.222.0.44 () {58 vars in 1361 bytes} [Sun Sep 11 08:24:34 2022] PUT /identity/v3/domains/default/users/e0cb001df8d24a87b7f9253f68745c35/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:24:34.590779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3765a-a619-4c74-9b4d-d938b20e8116 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:34.631380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22ceba-d8e3-4799-8986-b5d964b981b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:34.632021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22ceba-d8e3-4799-8986-b5d964b981b1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:34.632298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22ceba-d8e3-4799-8986-b5d964b981b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:34.632566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22ceba-d8e3-4799-8986-b5d964b981b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:34.633685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22ceba-d8e3-4799-8986-b5d964b981b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:34.635480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22ceba-d8e3-4799-8986-b5d964b981b1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:34.637151 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 23/45] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:24:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:24:34.643319 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2c749-88fe-43cf-af0e-c1c240455c5f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:34.712223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:34.780445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2c749-88fe-43cf-af0e-c1c240455c5f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:34.850452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007c4a1-13cb-4da0-905a-a6fcb63f957a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:34.851426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007c4a1-13cb-4da0-905a-a6fcb63f957a None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:34.851923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007c4a1-13cb-4da0-905a-a6fcb63f957a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:34.852424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007c4a1-13cb-4da0-905a-a6fcb63f957a None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:34.853775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007c4a1-13cb-4da0-905a-a6fcb63f957a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:34.856453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007c4a1-13cb-4da0-905a-a6fcb63f957a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:34.956642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 23/46] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:24:34 2022] POST /identity/v3/projects => generated 312 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:24:36.821342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc9e43-b036-412f-ad52-f2f4c7c974b1 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:36.821862 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc9e43-b036-412f-ad52-f2f4c7c974b1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:36.821862 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc9e43-b036-412f-ad52-f2f4c7c974b1 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:36.823306 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 24/47] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:36.828735 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8095224-e8e7-42fe-8881-03f74d835d57 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:36.829142 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8095224-e8e7-42fe-8881-03f74d835d57 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:36.829431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8095224-e8e7-42fe-8881-03f74d835d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:36.847945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b8095224-e8e7-42fe-8881-03f74d835d57 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:36.849172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b8095224-e8e7-42fe-8881-03f74d835d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:36.918205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 24/48] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:37.184128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78bf4a-bc52-4043-9044-9709e1382a27 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:37.184423 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78bf4a-bc52-4043-9044-9709e1382a27 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:37.184781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78bf4a-bc52-4043-9044-9709e1382a27 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:37.186239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 25/49] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:37.189930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8095224-e8e7-42fe-8881-03f74d835d57 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:37.191229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c66339-efee-4ddb-9451-1a0b91b1bb00 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:37.191526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c66339-efee-4ddb-9451-1a0b91b1bb00 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:37.191801 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c66339-efee-4ddb-9451-1a0b91b1bb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:37.210349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-46c66339-efee-4ddb-9451-1a0b91b1bb00 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:37.211900 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-46c66339-efee-4ddb-9451-1a0b91b1bb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:37.220184 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:37.279549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 26/50] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:37.290137 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8095224-e8e7-42fe-8881-03f74d835d57 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:37.325399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e895cbf-1bdc-4fff-b98a-13ae8cf4ccb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:37.326056 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e895cbf-1bdc-4fff-b98a-13ae8cf4ccb8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:37.326365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e895cbf-1bdc-4fff-b98a-13ae8cf4ccb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:37.326637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e895cbf-1bdc-4fff-b98a-13ae8cf4ccb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:37.339220 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e895cbf-1bdc-4fff-b98a-13ae8cf4ccb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1e895cbf-1bdc-4fff-b98a-13ae8cf4ccb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:24:37.344368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:37.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:37.346078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:37.346078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:37.346078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:37.346078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:37.346078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:24:37.346078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:24:37.346078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:24:37.346078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:24:37.346078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:24:37.346078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:24:37.346078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:24:37.346078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:24:37.346078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 25/51] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:24:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:24:37.352299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c66339-efee-4ddb-9451-1a0b91b1bb00 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:37.404079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:37.453705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c66339-efee-4ddb-9451-1a0b91b1bb00 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:37.489920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1522e4-266b-48b4-950f-5c3265d5eea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:37.490565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1522e4-266b-48b4-950f-5c3265d5eea4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:37.490843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1522e4-266b-48b4-950f-5c3265d5eea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:37.491112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1522e4-266b-48b4-950f-5c3265d5eea4 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:37.491778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1522e4-266b-48b4-950f-5c3265d5eea4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:37.492076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1522e4-266b-48b4-950f-5c3265d5eea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:37.493465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1522e4-266b-48b4-950f-5c3265d5eea4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:37.502401 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 27/52] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:24:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:37.508147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e895cbf-1bdc-4fff-b98a-13ae8cf4ccb8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:37.554045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1522e4-266b-48b4-950f-5c3265d5eea4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:37.608239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:37.616613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e895cbf-1bdc-4fff-b98a-13ae8cf4ccb8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:37.651282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c0ac8-274f-4e83-bc04-e0b0f49ce6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:37.652369 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c0ac8-274f-4e83-bc04-e0b0f49ce6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:37.652875 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c0ac8-274f-4e83-bc04-e0b0f49ce6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:37.653163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c0ac8-274f-4e83-bc04-e0b0f49ce6b7 None admin] PATH_INFO: `/v3/users/e0cb001df8d24a87b7f9253f68745c35` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:37.654369 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c0ac8-274f-4e83-bc04-e0b0f49ce6b7 None admin] RBAC: Authorizing `identity:get_user(user_id=e0cb001df8d24a87b7f9253f68745c35)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:37.655846 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c0ac8-274f-4e83-bc04-e0b0f49ce6b7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:37.658077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 26/53] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:24:37 2022] GET /identity/v3/users/e0cb001df8d24a87b7f9253f68745c35 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:37.662038 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1522e4-266b-48b4-950f-5c3265d5eea4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:37.664262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c0ac8-274f-4e83-bc04-e0b0f49ce6b7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:37.678948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:37.705905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da005c5-cbec-4b5b-8fe0-33fce99a3a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:37.706505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da005c5-cbec-4b5b-8fe0-33fce99a3a02 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:37.706780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da005c5-cbec-4b5b-8fe0-33fce99a3a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:37.707048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da005c5-cbec-4b5b-8fe0-33fce99a3a02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:37.708710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da005c5-cbec-4b5b-8fe0-33fce99a3a02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:37.710198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da005c5-cbec-4b5b-8fe0-33fce99a3a02 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:37.711935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 28/54] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:24:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:24:37.717465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da005c5-cbec-4b5b-8fe0-33fce99a3a02 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:37.768017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c0ac8-274f-4e83-bc04-e0b0f49ce6b7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:37.802274 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629dc0e-c2f5-41ca-89df-95bff03f6465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:37.802912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629dc0e-c2f5-41ca-89df-95bff03f6465 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:37.803193 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629dc0e-c2f5-41ca-89df-95bff03f6465 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:37.803464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629dc0e-c2f5-41ca-89df-95bff03f6465 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:37.804587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629dc0e-c2f5-41ca-89df-95bff03f6465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:37.806063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629dc0e-c2f5-41ca-89df-95bff03f6465 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:37.807673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 27/55] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:24:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:24:37.813619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629dc0e-c2f5-41ca-89df-95bff03f6465 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:37.819481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da005c5-cbec-4b5b-8fe0-33fce99a3a02 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:37.846970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:37.864194 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e769c4-2196-466c-8445-7ecf0f531099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:37.864869 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e769c4-2196-466c-8445-7ecf0f531099 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:37.865131 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e769c4-2196-466c-8445-7ecf0f531099 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:37.865412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e769c4-2196-466c-8445-7ecf0f531099 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:37.866073 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e769c4-2196-466c-8445-7ecf0f531099 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:37.867621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e769c4-2196-466c-8445-7ecf0f531099 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:37.926134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629dc0e-c2f5-41ca-89df-95bff03f6465 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:37.962244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e038a-478d-495a-a761-ee545a5e5f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:37.962690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e038a-478d-495a-a761-ee545a5e5f39 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:37.963116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e038a-478d-495a-a761-ee545a5e5f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:37.963229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e038a-478d-495a-a761-ee545a5e5f39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:37.964330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e038a-478d-495a-a761-ee545a5e5f39 None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.domain.id=default, user.id=e0cb001df8d24a87b7f9253f68745c35) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:37.964525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e038a-478d-495a-a761-ee545a5e5f39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:37.965983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e038a-478d-495a-a761-ee545a5e5f39 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:37.972965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 28/56] 10.222.0.44 () {56 vars in 1322 bytes} [Sun Sep 11 08:24:37 2022] GET /identity/v3/role_assignments?user.id=e0cb001df8d24a87b7f9253f68745c35&scope.domain.id=default&role.id=020bc02dca71485bab2312442adaa179 => generated 530 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:24:37.989551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 29/57] 10.222.0.44 () {60 vars in 1140 bytes} [Sun Sep 11 08:24:37 2022] POST /identity/v3/projects => generated 298 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:24:40.199010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2fbec-93c5-4fe9-88b6-b0084cf8db2c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:40.199502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2fbec-93c5-4fe9-88b6-b0084cf8db2c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:40.199502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2fbec-93c5-4fe9-88b6-b0084cf8db2c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:40.200924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 29/58] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:40.206610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8d9ae-abc7-4dcb-b38a-252f251c5ea0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:40.207060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8d9ae-abc7-4dcb-b38a-252f251c5ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:40.207197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8d9ae-abc7-4dcb-b38a-252f251c5ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:40.227054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a5b8d9ae-abc7-4dcb-b38a-252f251c5ea0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:40.228241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a5b8d9ae-abc7-4dcb-b38a-252f251c5ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:40.252797 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:40.306129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 30/59] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:40.572351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2fbec-93c5-4fe9-88b6-b0084cf8db2c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:40.667711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:40.677077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2fbec-93c5-4fe9-88b6-b0084cf8db2c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:40.712728 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749a8f6-1387-4632-8245-be7413ac89ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:40.713362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749a8f6-1387-4632-8245-be7413ac89ae None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:40.713902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749a8f6-1387-4632-8245-be7413ac89ae None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:40.714198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749a8f6-1387-4632-8245-be7413ac89ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:40.715316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749a8f6-1387-4632-8245-be7413ac89ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:40.716775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749a8f6-1387-4632-8245-be7413ac89ae None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:40.718387 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 30/60] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:24:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:40.724594 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8d9ae-abc7-4dcb-b38a-252f251c5ea0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:40.932150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8d9ae-abc7-4dcb-b38a-252f251c5ea0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:40.965080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6a451-3b43-42f2-9ad9-0e6e095d11a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:40.965691 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6a451-3b43-42f2-9ad9-0e6e095d11a9 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:40.965974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6a451-3b43-42f2-9ad9-0e6e095d11a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:40.966278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6a451-3b43-42f2-9ad9-0e6e095d11a9 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:40.966929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6a451-3b43-42f2-9ad9-0e6e095d11a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:40.968387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6a451-3b43-42f2-9ad9-0e6e095d11a9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:41.073013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 31/61] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:24:40 2022] POST /identity/v3/users => generated 275 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:24:44.821382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b871b-3150-483a-b9f6-ba2d259bbec6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:44.822432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b871b-3150-483a-b9f6-ba2d259bbec6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:44.822949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b871b-3150-483a-b9f6-ba2d259bbec6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:44.824538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 31/62] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:44.830318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77855e14-fa1e-4415-bbd9-ea209ca935fa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:44.831223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77855e14-fa1e-4415-bbd9-ea209ca935fa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:44.831707 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77855e14-fa1e-4415-bbd9-ea209ca935fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:44.851585 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-77855e14-fa1e-4415-bbd9-ea209ca935fa None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:44.853132 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-77855e14-fa1e-4415-bbd9-ea209ca935fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:44.879482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:44.942709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 32/63] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:45.140766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e765bb7-8157-4ead-b4dd-69501153943e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:45.141075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e765bb7-8157-4ead-b4dd-69501153943e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:45.141693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e765bb7-8157-4ead-b4dd-69501153943e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:45.143988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 32/64] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:45.157310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62513934-4e5e-4550-831f-0656f3bce1f0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:45.157827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62513934-4e5e-4550-831f-0656f3bce1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:45.158268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62513934-4e5e-4550-831f-0656f3bce1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:45.187191 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-62513934-4e5e-4550-831f-0656f3bce1f0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:45.188676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-62513934-4e5e-4550-831f-0656f3bce1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:45.271679 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 33/65] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:45.417908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e765bb7-8157-4ead-b4dd-69501153943e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:45.502228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a1681-bc03-45c3-836f-cb01dcd6502b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:45.502773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a1681-bc03-45c3-836f-cb01dcd6502b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:45.503196 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a1681-bc03-45c3-836f-cb01dcd6502b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:45.504747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 34/66] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:24:45.512667 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb633520-8c25-4f46-b6ba-79e0deaa96b7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:45.513169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb633520-8c25-4f46-b6ba-79e0deaa96b7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:45.513597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb633520-8c25-4f46-b6ba-79e0deaa96b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:45.544755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fb633520-8c25-4f46-b6ba-79e0deaa96b7 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:45.546771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fb633520-8c25-4f46-b6ba-79e0deaa96b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:45.659306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e765bb7-8157-4ead-b4dd-69501153943e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:45.704071 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 35/67] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:24:45.705271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb633520-8c25-4f46-b6ba-79e0deaa96b7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:45.720303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:45.763932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a238c70-9720-4770-b7c0-f5c63cb4bf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:45.764712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a238c70-9720-4770-b7c0-f5c63cb4bf3d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:45.765076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a238c70-9720-4770-b7c0-f5c63cb4bf3d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:45.765418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a238c70-9720-4770-b7c0-f5c63cb4bf3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:45.797535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a238c70-9720-4770-b7c0-f5c63cb4bf3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9a238c70-9720-4770-b7c0-f5c63cb4bf3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:24:45.802910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:45.804860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:45.806674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:45.806674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:45.806674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:45.806674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:45.806674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:24:45.806674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:24:45.806674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:24:45.806674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:24:45.806674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:24:45.806674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:24:45.806674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:24:45.806674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:24:45.806674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 33/68] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:24:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:24:45.816158 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a238c70-9720-4770-b7c0-f5c63cb4bf3d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:45.844068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:45.908078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb633520-8c25-4f46-b6ba-79e0deaa96b7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:45.971972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed9308-bfc5-436d-a95a-5f1d0e016678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:45.973263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed9308-bfc5-436d-a95a-5f1d0e016678 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:45.974491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed9308-bfc5-436d-a95a-5f1d0e016678 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:45.975129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed9308-bfc5-436d-a95a-5f1d0e016678 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:45.996486 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed9308-bfc5-436d-a95a-5f1d0e016678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3bed9308-bfc5-436d-a95a-5f1d0e016678 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:24:46.005803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:46.006811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:46.007577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:46.007577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:46.007577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:46.007577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:46.007577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:24:46.007577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:24:46.007577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:24:46.007577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:24:46.007577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:24:46.007577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:24:46.007577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:24:46.007577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:24:46.008145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a238c70-9720-4770-b7c0-f5c63cb4bf3d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:46.008213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 36/69] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:24:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:24:46.011834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9098c-45e4-4820-b458-c74409ca1026 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:46.012573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9098c-45e4-4820-b458-c74409ca1026 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:46.013203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9098c-45e4-4820-b458-c74409ca1026 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:46.015868 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 37/70] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:46.019117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9098c-45e4-4820-b458-c74409ca1026 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:46.057089 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525faf42-396e-4269-b539-347749ffbbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:46.058022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525faf42-396e-4269-b539-347749ffbbf5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:46.058430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525faf42-396e-4269-b539-347749ffbbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:46.058870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525faf42-396e-4269-b539-347749ffbbf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:46.059690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525faf42-396e-4269-b539-347749ffbbf5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:46.059994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525faf42-396e-4269-b539-347749ffbbf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:46.061944 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525faf42-396e-4269-b539-347749ffbbf5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:46.078128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 34/71] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:24:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:24:46.078128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525faf42-396e-4269-b539-347749ffbbf5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:46.103196 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:46.143416 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9098c-45e4-4820-b458-c74409ca1026 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:46.169404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:46.185694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97609e96-a89f-4594-95f3-36ab9156bd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:46.186655 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97609e96-a89f-4594-95f3-36ab9156bd1b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:46.187064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97609e96-a89f-4594-95f3-36ab9156bd1b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:46.187479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97609e96-a89f-4594-95f3-36ab9156bd1b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:46.189189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525faf42-396e-4269-b539-347749ffbbf5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:46.200023 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97609e96-a89f-4594-95f3-36ab9156bd1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-97609e96-a89f-4594-95f3-36ab9156bd1b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:24:46.205022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:46.205880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:46.206685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:46.206685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:46.206685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:46.206685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:46.206685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:24:46.206685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:24:46.206685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:24:46.206685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:24:46.206685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:24:46.206685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:24:46.206685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:24:46.206685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:24:46.206685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 38/72] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:24:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:24:46.207713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a3785-903d-42f9-8946-17dc3d74715e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:46.208519 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a3785-903d-42f9-8946-17dc3d74715e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:46.208793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a3785-903d-42f9-8946-17dc3d74715e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:46.229124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-490a3785-903d-42f9-8946-17dc3d74715e None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:46.230318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-490a3785-903d-42f9-8946-17dc3d74715e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:46.232058 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00febc3f-d8c7-4a9b-bb6b-c41eb2e0e0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:46.232710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00febc3f-d8c7-4a9b-bb6b-c41eb2e0e0aa None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:46.232990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00febc3f-d8c7-4a9b-bb6b-c41eb2e0e0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:46.233272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00febc3f-d8c7-4a9b-bb6b-c41eb2e0e0aa None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:46.234098 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00febc3f-d8c7-4a9b-bb6b-c41eb2e0e0aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:46.234456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00febc3f-d8c7-4a9b-bb6b-c41eb2e0e0aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:46.235905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00febc3f-d8c7-4a9b-bb6b-c41eb2e0e0aa None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:46.245988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 35/73] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:24:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:46.247362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e704a2c-e78b-411c-94eb-08387956cf29 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:46.247725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e704a2c-e78b-411c-94eb-08387956cf29 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:46.248008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e704a2c-e78b-411c-94eb-08387956cf29 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:46.249423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 36/74] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:46.251537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e704a2c-e78b-411c-94eb-08387956cf29 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:46.329551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 39/75] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:46.330692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a3785-903d-42f9-8946-17dc3d74715e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:46.405791 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:46.424375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e704a2c-e78b-411c-94eb-08387956cf29 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:46.458639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a3785-903d-42f9-8946-17dc3d74715e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:46.486790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22291cb-d3ed-4642-8c9c-5344609cddb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:46.488010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22291cb-d3ed-4642-8c9c-5344609cddb5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:46.488400 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22291cb-d3ed-4642-8c9c-5344609cddb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:46.488818 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22291cb-d3ed-4642-8c9c-5344609cddb5 None admin] PATH_INFO: `/v3/users/e0cb001df8d24a87b7f9253f68745c35` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:46.490898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22291cb-d3ed-4642-8c9c-5344609cddb5 None admin] RBAC: Authorizing `identity:get_user(user_id=e0cb001df8d24a87b7f9253f68745c35)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:46.493609 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22291cb-d3ed-4642-8c9c-5344609cddb5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:46.496724 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 37/76] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:24:46 2022] GET /identity/v3/users/e0cb001df8d24a87b7f9253f68745c35 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:46.498409 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22291cb-d3ed-4642-8c9c-5344609cddb5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:46.509405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34316555-2f8a-4dd4-907a-80b490134fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:46.510035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34316555-2f8a-4dd4-907a-80b490134fbe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:46.510352 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34316555-2f8a-4dd4-907a-80b490134fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:46.510628 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34316555-2f8a-4dd4-907a-80b490134fbe None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:46.511313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34316555-2f8a-4dd4-907a-80b490134fbe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:46.511619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34316555-2f8a-4dd4-907a-80b490134fbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:46.513697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34316555-2f8a-4dd4-907a-80b490134fbe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:46.521321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:46.531425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 40/77] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:24:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:46.531425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb15e30-e7b1-4265-8443-66f072b74433 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:46.531425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb15e30-e7b1-4265-8443-66f072b74433 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:46.531425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb15e30-e7b1-4265-8443-66f072b74433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:46.551412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0cb15e30-e7b1-4265-8443-66f072b74433 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:46.552871 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0cb15e30-e7b1-4265-8443-66f072b74433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:46.625735 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22291cb-d3ed-4642-8c9c-5344609cddb5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:46.643994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 41/78] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:46.645168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb15e30-e7b1-4265-8443-66f072b74433 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:46.686440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c5c6a-6053-4b51-bfc8-1fd5a16bfb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:46.687444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c5c6a-6053-4b51-bfc8-1fd5a16bfb7f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:46.687899 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c5c6a-6053-4b51-bfc8-1fd5a16bfb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:46.688352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c5c6a-6053-4b51-bfc8-1fd5a16bfb7f None admin] PATH_INFO: `/v3/users/107dc688e53144dea3e37d8737d74742` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:46.711946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c5c6a-6053-4b51-bfc8-1fd5a16bfb7f None admin] RBAC: Authorizing `identity:get_user(user_id=107dc688e53144dea3e37d8737d74742)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:46.714829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c5c6a-6053-4b51-bfc8-1fd5a16bfb7f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:46.717549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 38/79] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:24:46 2022] GET /identity/v3/users/107dc688e53144dea3e37d8737d74742 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:46.718907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c5c6a-6053-4b51-bfc8-1fd5a16bfb7f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:46.736238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:46.828037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb15e30-e7b1-4265-8443-66f072b74433 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:46.850252 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:46.878067 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cf98e-da21-436c-a2f5-bd01a70014c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:46.879109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cf98e-da21-436c-a2f5-bd01a70014c4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:46.879441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cf98e-da21-436c-a2f5-bd01a70014c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:46.880234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cf98e-da21-436c-a2f5-bd01a70014c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:46.904536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c5c6a-6053-4b51-bfc8-1fd5a16bfb7f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:46.954439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cb642-be2d-4f34-9436-7d0e306f01e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:46.955547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cb642-be2d-4f34-9436-7d0e306f01e0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:46.956047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cb642-be2d-4f34-9436-7d0e306f01e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:46.956432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cb642-be2d-4f34-9436-7d0e306f01e0 None admin] PATH_INFO: `/v3/users/107dc688e53144dea3e37d8737d74742` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:46.958897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cb642-be2d-4f34-9436-7d0e306f01e0 None admin] RBAC: Authorizing `identity:get_user(user_id=107dc688e53144dea3e37d8737d74742)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:46.959974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cf98e-da21-436c-a2f5-bd01a70014c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:46.960420 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cb642-be2d-4f34-9436-7d0e306f01e0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:46.961446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cf98e-da21-436c-a2f5-bd01a70014c4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:46.962026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 39/80] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:24:46 2022] GET /identity/v3/users/107dc688e53144dea3e37d8737d74742 => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:46.962847 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cb642-be2d-4f34-9436-7d0e306f01e0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:47.017193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:47.071889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 42/81] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:24:47.073001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cf98e-da21-436c-a2f5-bd01a70014c4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:47.080247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cb642-be2d-4f34-9436-7d0e306f01e0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:47.127342 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf20eb-caf4-4a74-afc2-5763c357bb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:47.128053 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf20eb-caf4-4a74-afc2-5763c357bb1a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:47.128404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf20eb-caf4-4a74-afc2-5763c357bb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:47.128676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf20eb-caf4-4a74-afc2-5763c357bb1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:47.129844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf20eb-caf4-4a74-afc2-5763c357bb1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:47.131369 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf20eb-caf4-4a74-afc2-5763c357bb1a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:47.134203 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 40/82] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:24:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:47.134715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf20eb-caf4-4a74-afc2-5763c357bb1a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:47.144839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:47.214195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cf98e-da21-436c-a2f5-bd01a70014c4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:47.250822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372590ed-352a-4e66-b788-772abc15a609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:47.251445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372590ed-352a-4e66-b788-772abc15a609 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:47.251728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372590ed-352a-4e66-b788-772abc15a609 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:47.252089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372590ed-352a-4e66-b788-772abc15a609 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:47.252770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf20eb-caf4-4a74-afc2-5763c357bb1a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:47.276974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:47.289690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d7d6c7-48e5-4144-9817-e7aeff622063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:47.290374 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7d6c7-48e5-4144-9817-e7aeff622063 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:47.290661 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7d6c7-48e5-4144-9817-e7aeff622063 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:47.290937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7d6c7-48e5-4144-9817-e7aeff622063 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:47.303380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d7d6c7-48e5-4144-9817-e7aeff622063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d3d7d6c7-48e5-4144-9817-e7aeff622063 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:24:47.308844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:47.310190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 41/83] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:24:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:24:47.311645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d7d6c7-48e5-4144-9817-e7aeff622063 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:47.348148 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372590ed-352a-4e66-b788-772abc15a609 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:47.350406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372590ed-352a-4e66-b788-772abc15a609 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:47.385493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:47.425078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d7d6c7-48e5-4144-9817-e7aeff622063 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:47.466105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 43/84] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:24:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:24:47.467006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372590ed-352a-4e66-b788-772abc15a609 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:47.482636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06dd98-2837-452f-abc1-a82d3237f1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:47.483264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06dd98-2837-452f-abc1-a82d3237f1ca None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:47.483563 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06dd98-2837-452f-abc1-a82d3237f1ca None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:47.483824 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06dd98-2837-452f-abc1-a82d3237f1ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:47.525702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:47.554427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06dd98-2837-452f-abc1-a82d3237f1ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:47.555913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06dd98-2837-452f-abc1-a82d3237f1ca None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:47.578673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372590ed-352a-4e66-b788-772abc15a609 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:47.625612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:47.626044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ae470-22d7-4929-a4a5-2d92c8d1c2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:47.626755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ae470-22d7-4929-a4a5-2d92c8d1c2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:47.627024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ae470-22d7-4929-a4a5-2d92c8d1c2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:47.627302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ae470-22d7-4929-a4a5-2d92c8d1c2d4 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:47.628834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ae470-22d7-4929-a4a5-2d92c8d1c2d4 None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:47.631475 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ae470-22d7-4929-a4a5-2d92c8d1c2d4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:47.633556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 44/85] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:24:47 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:47.634289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ae470-22d7-4929-a4a5-2d92c8d1c2d4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:47.681407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 42/86] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:24:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:24:47.682457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06dd98-2837-452f-abc1-a82d3237f1ca None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:47.781501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ae470-22d7-4929-a4a5-2d92c8d1c2d4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:47.801858 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06dd98-2837-452f-abc1-a82d3237f1ca None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:47.807138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:47.826432 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417893a7-0d61-4476-bf5d-596b9d5f5ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:47.827049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417893a7-0d61-4476-bf5d-596b9d5f5ffe None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:47.827320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417893a7-0d61-4476-bf5d-596b9d5f5ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:47.827588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417893a7-0d61-4476-bf5d-596b9d5f5ffe None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:47.828251 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417893a7-0d61-4476-bf5d-596b9d5f5ffe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:47.829721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417893a7-0d61-4476-bf5d-596b9d5f5ffe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:47.854244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5edf2-a19e-43a2-a908-9caaf27cda0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:47.855312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5edf2-a19e-43a2-a908-9caaf27cda0a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:47.855797 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5edf2-a19e-43a2-a908-9caaf27cda0a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:47.856167 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5edf2-a19e-43a2-a908-9caaf27cda0a None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:47.857355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5edf2-a19e-43a2-a908-9caaf27cda0a None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:47.857882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5edf2-a19e-43a2-a908-9caaf27cda0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:47.860249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c5edf2-a19e-43a2-a908-9caaf27cda0a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:47.881214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 43/87] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:24:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:47.881214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5edf2-a19e-43a2-a908-9caaf27cda0a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:47.925813 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:47.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 45/88] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:24:47 2022] POST /identity/v3/users => generated 289 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:24:47.937556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417893a7-0d61-4476-bf5d-596b9d5f5ffe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:48.069474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5edf2-a19e-43a2-a908-9caaf27cda0a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:48.123177 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fc5c3-ec4c-4172-b549-121c5d9e80b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:48.124155 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fc5c3-ec4c-4172-b549-121c5d9e80b7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:48.124575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fc5c3-ec4c-4172-b549-121c5d9e80b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:48.124976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fc5c3-ec4c-4172-b549-121c5d9e80b7 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:48.126709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fc5c3-ec4c-4172-b549-121c5d9e80b7 None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:48.128680 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fc5c3-ec4c-4172-b549-121c5d9e80b7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:48.137265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 44/89] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:24:47 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:48.137432 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417893a7-0d61-4476-bf5d-596b9d5f5ffe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:48.137879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fc5c3-ec4c-4172-b549-121c5d9e80b7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:48.197153 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3501fb-dcfc-4798-b8fb-ef753418941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:48.198078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3501fb-dcfc-4798-b8fb-ef753418941a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:48.198387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3501fb-dcfc-4798-b8fb-ef753418941a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:48.198676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3501fb-dcfc-4798-b8fb-ef753418941a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:48.199443 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3501fb-dcfc-4798-b8fb-ef753418941a None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7, user.id=e0cb001df8d24a87b7f9253f68745c35) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:48.199755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3501fb-dcfc-4798-b8fb-ef753418941a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:48.201257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3501fb-dcfc-4798-b8fb-ef753418941a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:48.212976 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 46/90] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:24:47 2022] GET /identity/v3/role_assignments?user.id=e0cb001df8d24a87b7f9253f68745c35&scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7&role.id=020bc02dca71485bab2312442adaa179 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:48.222296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3501fb-dcfc-4798-b8fb-ef753418941a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:48.278423 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:48.314052 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:48.321549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fc5c3-ec4c-4172-b549-121c5d9e80b7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:48.390738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1974917-9a4e-43b3-beb2-223501afa632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:48.391678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1974917-9a4e-43b3-beb2-223501afa632 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:48.392157 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1974917-9a4e-43b3-beb2-223501afa632 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:48.392631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1974917-9a4e-43b3-beb2-223501afa632 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:48.394667 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1974917-9a4e-43b3-beb2-223501afa632 None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:48.397309 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1974917-9a4e-43b3-beb2-223501afa632 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:48.400341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 45/91] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:24:48 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:48.401847 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1974917-9a4e-43b3-beb2-223501afa632 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:48.431098 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3501fb-dcfc-4798-b8fb-ef753418941a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:48.499464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c106f-3c6d-4a16-b4b8-28d90ab8a4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:48.500491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c106f-3c6d-4a16-b4b8-28d90ab8a4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:48.500953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c106f-3c6d-4a16-b4b8-28d90ab8a4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:48.501408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c106f-3c6d-4a16-b4b8-28d90ab8a4f6 None admin] PATH_INFO: `/v3/users/107dc688e53144dea3e37d8737d74742` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:48.507650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c106f-3c6d-4a16-b4b8-28d90ab8a4f6 None admin] RBAC: Authorizing `identity:get_user(user_id=107dc688e53144dea3e37d8737d74742)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:48.509166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c106f-3c6d-4a16-b4b8-28d90ab8a4f6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:48.511284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 47/92] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:24:48 2022] GET /identity/v3/users/107dc688e53144dea3e37d8737d74742 => generated 275 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:48.512134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c106f-3c6d-4a16-b4b8-28d90ab8a4f6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:48.674437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1974917-9a4e-43b3-beb2-223501afa632 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:48.799207 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c106f-3c6d-4a16-b4b8-28d90ab8a4f6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:48.815407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0046a6-41be-4092-8bfd-6e426daaee05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:48.816438 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0046a6-41be-4092-8bfd-6e426daaee05 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:48.816928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0046a6-41be-4092-8bfd-6e426daaee05 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:48.817993 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0046a6-41be-4092-8bfd-6e426daaee05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:48.818753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:48.819384 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0046a6-41be-4092-8bfd-6e426daaee05 None admin] RBAC: Adding query filter params (role.id=dba75ce7d709483fbbb0027d29e99725, scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7, user.id=107dc688e53144dea3e37d8737d74742) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:48.820010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0046a6-41be-4092-8bfd-6e426daaee05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:48.822826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0046a6-41be-4092-8bfd-6e426daaee05 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:48.845668 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 46/93] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:24:48 2022] GET /identity/v3/role_assignments?user.id=107dc688e53144dea3e37d8737d74742&scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7&role.id=dba75ce7d709483fbbb0027d29e99725 => generated 259 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:48.847217 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0046a6-41be-4092-8bfd-6e426daaee05 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:48.877693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccc52f-124c-47a2-842d-dd07fc51ac97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:48.879160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccc52f-124c-47a2-842d-dd07fc51ac97 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:48.880599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccc52f-124c-47a2-842d-dd07fc51ac97 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:48.881318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccc52f-124c-47a2-842d-dd07fc51ac97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:48.883150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccc52f-124c-47a2-842d-dd07fc51ac97 None admin] RBAC: Adding query filter params (role.id=65f33fe3045c433b9a3e045e7572aed6, scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7, user.id=107dc688e53144dea3e37d8737d74742) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:48.884034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccc52f-124c-47a2-842d-dd07fc51ac97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:48.887145 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccc52f-124c-47a2-842d-dd07fc51ac97 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:48.898560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 48/94] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:24:48 2022] GET /identity/v3/role_assignments?user.id=107dc688e53144dea3e37d8737d74742&scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7&role.id=65f33fe3045c433b9a3e045e7572aed6 => generated 259 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:48.949793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:49.068823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0046a6-41be-4092-8bfd-6e426daaee05 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:49.125455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104c510-9da5-4904-aa7b-e3c6fa109f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:49.126169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104c510-9da5-4904-aa7b-e3c6fa109f9b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:49.126412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104c510-9da5-4904-aa7b-e3c6fa109f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:49.126683 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104c510-9da5-4904-aa7b-e3c6fa109f9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:49.294772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104c510-9da5-4904-aa7b-e3c6fa109f9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:49.298191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104c510-9da5-4904-aa7b-e3c6fa109f9b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:49.358634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:49.476942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 47/95] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:24:49.483387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccc52f-124c-47a2-842d-dd07fc51ac97 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:49.702808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccc52f-124c-47a2-842d-dd07fc51ac97 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:49.766503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320a590-dfba-409c-8216-f09685927a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:49.767992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320a590-dfba-409c-8216-f09685927a31 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:49.768724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320a590-dfba-409c-8216-f09685927a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:49.769418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320a590-dfba-409c-8216-f09685927a31 None admin] PATH_INFO: `/v3/projects/2d2d9cc801e64f22b0116db19cb339f0` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:49.772324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320a590-dfba-409c-8216-f09685927a31 None admin] RBAC: Authorizing `identity:get_project(project_id=2d2d9cc801e64f22b0116db19cb339f0)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:49.775286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320a590-dfba-409c-8216-f09685927a31 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:49.779063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 49/96] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:24:49 2022] GET /identity/v3/projects/2d2d9cc801e64f22b0116db19cb339f0 => generated 312 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:49.785388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104c510-9da5-4904-aa7b-e3c6fa109f9b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:49.965569 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104c510-9da5-4904-aa7b-e3c6fa109f9b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:50.004128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:50.013647 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86c859-5f33-4371-94cc-81c814b2ebe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:50.014315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86c859-5f33-4371-94cc-81c814b2ebe7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:50.014598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86c859-5f33-4371-94cc-81c814b2ebe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:50.025525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86c859-5f33-4371-94cc-81c814b2ebe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:50.025525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86c859-5f33-4371-94cc-81c814b2ebe7 None admin] RBAC: Adding query filter params (role.id=dba75ce7d709483fbbb0027d29e99725, scope.project.id=2d2d9cc801e64f22b0116db19cb339f0, user.id=107dc688e53144dea3e37d8737d74742) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:50.025525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86c859-5f33-4371-94cc-81c814b2ebe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:50.025525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86c859-5f33-4371-94cc-81c814b2ebe7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:50.027336 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 48/97] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:24:49 2022] GET /identity/v3/role_assignments?user.id=107dc688e53144dea3e37d8737d74742&scope.project.id=2d2d9cc801e64f22b0116db19cb339f0&role.id=dba75ce7d709483fbbb0027d29e99725 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:52.314688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f83159-0ea1-44f2-9428-4b16a260446f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:52.315432 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f83159-0ea1-44f2-9428-4b16a260446f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:52.315834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f83159-0ea1-44f2-9428-4b16a260446f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:52.318853 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 50/98] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:52.325639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd93cc-4992-4929-98bb-163638e36638 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:52.326404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd93cc-4992-4929-98bb-163638e36638 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:52.326718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd93cc-4992-4929-98bb-163638e36638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:52.341806 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f861df-00de-4b60-8202-524f7467efc2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:52.342314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f861df-00de-4b60-8202-524f7467efc2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:52.342813 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f861df-00de-4b60-8202-524f7467efc2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:52.345383 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 51/99] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:52.352107 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751c10-fd1c-4fb7-9e27-344a2ab2fcef None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:52.352518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751c10-fd1c-4fb7-9e27-344a2ab2fcef None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:52.352928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751c10-fd1c-4fb7-9e27-344a2ab2fcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:52.404299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-50fd93cc-4992-4929-98bb-163638e36638 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:52.406715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-50fd93cc-4992-4929-98bb-163638e36638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:52.415639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f2751c10-fd1c-4fb7-9e27-344a2ab2fcef None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:52.417967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f2751c10-fd1c-4fb7-9e27-344a2ab2fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:52.538257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:52.661521 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 52/100] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:52.686533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 49/101] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:52.759700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d3b6f-c1f0-4027-93ef-d895c424e8f3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:52.760384 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d3b6f-c1f0-4027-93ef-d895c424e8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:52.760992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d3b6f-c1f0-4027-93ef-d895c424e8f3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:52.763637 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 53/102] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:52.773307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43dc62-7857-4ad1-bddb-4560780d56c5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:52.773972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43dc62-7857-4ad1-bddb-4560780d56c5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:52.774504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43dc62-7857-4ad1-bddb-4560780d56c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:52.823511 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-af43dc62-7857-4ad1-bddb-4560780d56c5 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:52.825555 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-af43dc62-7857-4ad1-bddb-4560780d56c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:52.999362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 50/103] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:53.017046 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d3b6f-c1f0-4027-93ef-d895c424e8f3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:53.096871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43dc62-7857-4ad1-bddb-4560780d56c5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:53.174755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:53.275566 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43dc62-7857-4ad1-bddb-4560780d56c5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:53.294956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d3b6f-c1f0-4027-93ef-d895c424e8f3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:53.358804 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486aefc-9c40-44a5-b5a0-96e9701e7d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:53.360402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486aefc-9c40-44a5-b5a0-96e9701e7d03 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:53.360897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486aefc-9c40-44a5-b5a0-96e9701e7d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:53.361379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486aefc-9c40-44a5-b5a0-96e9701e7d03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:53.366038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486aefc-9c40-44a5-b5a0-96e9701e7d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:53.368087 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486aefc-9c40-44a5-b5a0-96e9701e7d03 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:53.370274 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 51/104] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:24:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:53.373239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420e6cb-57c5-4b52-bf1d-c274157338c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:53.373239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420e6cb-57c5-4b52-bf1d-c274157338c0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:53.373239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420e6cb-57c5-4b52-bf1d-c274157338c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:53.373239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420e6cb-57c5-4b52-bf1d-c274157338c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:53.377614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486aefc-9c40-44a5-b5a0-96e9701e7d03 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:53.401332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420e6cb-57c5-4b52-bf1d-c274157338c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2420e6cb-57c5-4b52-bf1d-c274157338c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:24:53.407374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:53.408326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:53.409147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:53.409147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:53.409147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:53.409147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:53.409147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:24:53.409147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:24:53.409147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:24:53.409147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:24:53.409147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:24:53.409147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:24:53.409147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:24:53.409147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:24:53.409147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 54/105] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:24:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:24:53.417939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f79c6c-679c-4c5c-a9d7-691bde292e0b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:53.418310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f79c6c-679c-4c5c-a9d7-691bde292e0b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:53.418649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f79c6c-679c-4c5c-a9d7-691bde292e0b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:53.420762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 55/106] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:53.421640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f79c6c-679c-4c5c-a9d7-691bde292e0b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:53.471850 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:53.567682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486aefc-9c40-44a5-b5a0-96e9701e7d03 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:53.604313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f79c6c-679c-4c5c-a9d7-691bde292e0b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:53.614870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:53.627093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f21c8-239a-4150-be02-5a540764d4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:53.628204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f21c8-239a-4150-be02-5a540764d4fd None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:53.647608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f21c8-239a-4150-be02-5a540764d4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:53.647608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f21c8-239a-4150-be02-5a540764d4fd None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:53.647608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f21c8-239a-4150-be02-5a540764d4fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:53.647608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f21c8-239a-4150-be02-5a540764d4fd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:53.654585 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8a753-ecd2-49e1-be7c-d91628f049cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:53.655697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8a753-ecd2-49e1-be7c-d91628f049cc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:53.656175 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8a753-ecd2-49e1-be7c-d91628f049cc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:53.656663 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8a753-ecd2-49e1-be7c-d91628f049cc None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:53.657928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8a753-ecd2-49e1-be7c-d91628f049cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:53.658508 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8a753-ecd2-49e1-be7c-d91628f049cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:53.661403 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8a753-ecd2-49e1-be7c-d91628f049cc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:53.678472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 56/107] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:24:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:53.682452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85c21a-cad1-424d-bd1d-8800564b9ccb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:53.682452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85c21a-cad1-424d-bd1d-8800564b9ccb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:53.682452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85c21a-cad1-424d-bd1d-8800564b9ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:53.721678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ed85c21a-cad1-424d-bd1d-8800564b9ccb None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:53.723865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ed85c21a-cad1-424d-bd1d-8800564b9ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:53.750317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 52/108] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:24:53 2022] POST /identity/v3/projects => generated 302 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:24:53.750317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f21c8-239a-4150-be02-5a540764d4fd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:53.862538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 57/109] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:53.864620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9900cb-316f-4312-8885-bc089621745c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:53.865079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9900cb-316f-4312-8885-bc089621745c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:53.865526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9900cb-316f-4312-8885-bc089621745c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:53.868350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 58/110] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:53.869879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9900cb-316f-4312-8885-bc089621745c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:53.914514 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:54.081132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f21c8-239a-4150-be02-5a540764d4fd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:54.094105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9900cb-316f-4312-8885-bc089621745c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:54.165441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917631d-4cdb-4058-ab0c-ff2e9cb402c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:54.167209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917631d-4cdb-4058-ab0c-ff2e9cb402c7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:54.168043 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917631d-4cdb-4058-ab0c-ff2e9cb402c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:54.168735 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917631d-4cdb-4058-ab0c-ff2e9cb402c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:54.169159 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7e5a5-4e55-4a40-a92a-72bb63177b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:54.170284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7e5a5-4e55-4a40-a92a-72bb63177b03 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:54.170910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7e5a5-4e55-4a40-a92a-72bb63177b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:54.171472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7e5a5-4e55-4a40-a92a-72bb63177b03 None admin] PATH_INFO: `/v3/users/e0cb001df8d24a87b7f9253f68745c35` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:54.173825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7e5a5-4e55-4a40-a92a-72bb63177b03 None admin] RBAC: Authorizing `identity:get_user(user_id=e0cb001df8d24a87b7f9253f68745c35)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:54.177752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7e5a5-4e55-4a40-a92a-72bb63177b03 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:54.180928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 59/111] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:24:53 2022] GET /identity/v3/users/e0cb001df8d24a87b7f9253f68745c35 => generated 247 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:54.184556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d73d41-c3e5-4d51-a3fd-0afed24e2533 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:54.185123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d73d41-c3e5-4d51-a3fd-0afed24e2533 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:54.185660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d73d41-c3e5-4d51-a3fd-0afed24e2533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:54.194874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917631d-4cdb-4058-ab0c-ff2e9cb402c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1917631d-4cdb-4058-ab0c-ff2e9cb402c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:24:54.204541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:54.206338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:54.207914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:54.207914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:54.207914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:54.207914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:54.207914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:24:54.207914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:24:54.207914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:24:54.207914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:24:54.207914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:24:54.207914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:24:54.207914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:24:54.207914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:24:54.211945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 53/112] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:24:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:24:54.213674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917631d-4cdb-4058-ab0c-ff2e9cb402c7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:54.221062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-99d73d41-c3e5-4d51-a3fd-0afed24e2533 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:54.224576 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-99d73d41-c3e5-4d51-a3fd-0afed24e2533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:54.298948 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:54.365118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 60/113] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:54.374201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d73d41-c3e5-4d51-a3fd-0afed24e2533 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:54.414023 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:54.480670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917631d-4cdb-4058-ab0c-ff2e9cb402c7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:54.564361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d9432-f949-402b-a207-9211c624ccd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:54.566245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d9432-f949-402b-a207-9211c624ccd1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:54.566749 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d9432-f949-402b-a207-9211c624ccd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:54.567500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d9432-f949-402b-a207-9211c624ccd1 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:54.570294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d9432-f949-402b-a207-9211c624ccd1 None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:54.574367 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d9432-f949-402b-a207-9211c624ccd1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:54.578059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 54/114] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:24:54 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:54.581719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9565d49-3641-489a-9edb-ba4d9f87e561 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:54.582236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9565d49-3641-489a-9edb-ba4d9f87e561 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:54.582763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9565d49-3641-489a-9edb-ba4d9f87e561 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:54.584819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d73d41-c3e5-4d51-a3fd-0afed24e2533 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:54.585365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 55/115] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:54.591734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9565d49-3641-489a-9edb-ba4d9f87e561 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:54.674888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c038ff9-f708-4805-8cf2-7f7c9f022a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:54.678195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c038ff9-f708-4805-8cf2-7f7c9f022a06 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:54.678710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c038ff9-f708-4805-8cf2-7f7c9f022a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:54.679188 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c038ff9-f708-4805-8cf2-7f7c9f022a06 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:54.680473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c038ff9-f708-4805-8cf2-7f7c9f022a06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:54.681009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c038ff9-f708-4805-8cf2-7f7c9f022a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:54.683842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c038ff9-f708-4805-8cf2-7f7c9f022a06 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:54.706389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 61/116] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:24:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:54.708980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c038ff9-f708-4805-8cf2-7f7c9f022a06 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:54.823034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:54.829474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9565d49-3641-489a-9edb-ba4d9f87e561 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:54.911945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:54.917887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4a137-52d0-4b60-a61c-7519b6339cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:54.919044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4a137-52d0-4b60-a61c-7519b6339cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:54.919508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4a137-52d0-4b60-a61c-7519b6339cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:54.919978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4a137-52d0-4b60-a61c-7519b6339cb9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:54.934646 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4a137-52d0-4b60-a61c-7519b6339cb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:24:54.941078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c038ff9-f708-4805-8cf2-7f7c9f022a06 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0ab4a137-52d0-4b60-a61c-7519b6339cb9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:24:54.944718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:54.945605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:54.946424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:54.946424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:54.946424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:54.946424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:54.946424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:24:54.946424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:24:54.946424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:24:54.946424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:24:54.946424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:24:54.946424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:24:54.946424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:24:54.946424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:24:54.947021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 56/117] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:24:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:24:54.948944 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa047c-f296-4050-acb4-0701ff0151f7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:54.949439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa047c-f296-4050-acb4-0701ff0151f7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:54.949922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa047c-f296-4050-acb4-0701ff0151f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:54.981619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a9fa047c-f296-4050-acb4-0701ff0151f7 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:54.983733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a9fa047c-f296-4050-acb4-0701ff0151f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:54.984880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce790e-f004-49f2-9776-5a4476c2daf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:54.985563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce790e-f004-49f2-9776-5a4476c2daf6 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:54.985847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce790e-f004-49f2-9776-5a4476c2daf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:54.986126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce790e-f004-49f2-9776-5a4476c2daf6 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7/users/e0cb001df8d24a87b7f9253f68745c35/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:54.988178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce790e-f004-49f2-9776-5a4476c2daf6 None admin] RBAC: Authorizing `identity:create_grant(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7, user_id=e0cb001df8d24a87b7f9253f68745c35, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:54.989700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce790e-f004-49f2-9776-5a4476c2daf6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:55.074293 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 62/118] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:24:54 2022] PUT /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7/users/e0cb001df8d24a87b7f9253f68745c35/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:24:55.074293 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce790e-f004-49f2-9776-5a4476c2daf6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:55.113857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 57/119] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:55.114659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa047c-f296-4050-acb4-0701ff0151f7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:55.294584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:55.300326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce790e-f004-49f2-9776-5a4476c2daf6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:55.314989 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa047c-f296-4050-acb4-0701ff0151f7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:55.408526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6bcf6-5a96-409b-8bdf-ba4ce67fa7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:55.408526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6bcf6-5a96-409b-8bdf-ba4ce67fa7fe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:55.408526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6bcf6-5a96-409b-8bdf-ba4ce67fa7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:55.408526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6bcf6-5a96-409b-8bdf-ba4ce67fa7fe None admin] PATH_INFO: `/v3/users/76ae9ead961542bba17e55eac70e16d2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:55.417475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ce872-f35e-4c08-b5e4-783c1779fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:55.418972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ce872-f35e-4c08-b5e4-783c1779fa6d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:55.419710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ce872-f35e-4c08-b5e4-783c1779fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:55.420416 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ce872-f35e-4c08-b5e4-783c1779fa6d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:55.422838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:55.428522 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6bcf6-5a96-409b-8bdf-ba4ce67fa7fe None admin] RBAC: Authorizing `identity:get_user(user_id=76ae9ead961542bba17e55eac70e16d2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:55.430082 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6bcf6-5a96-409b-8bdf-ba4ce67fa7fe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:55.432414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 63/120] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:24:55 2022] GET /identity/v3/users/76ae9ead961542bba17e55eac70e16d2 => generated 289 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:55.433316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6bcf6-5a96-409b-8bdf-ba4ce67fa7fe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:55.450196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ce872-f35e-4c08-b5e4-783c1779fa6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e10ce872-f35e-4c08-b5e4-783c1779fa6d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:24:55.461968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:55.464136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:55.466034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:55.466034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:55.466034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:55.466034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:55.466034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:24:55.466034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:24:55.466034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:24:55.466034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:24:55.466034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:24:55.466034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:24:55.466034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:24:55.466034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:24:55.470434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 58/121] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:24:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:24:55.472316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ce872-f35e-4c08-b5e4-783c1779fa6d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:55.555032 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6bcf6-5a96-409b-8bdf-ba4ce67fa7fe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:55.595578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0a665-de58-4485-83b6-2ae178018a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:55.596198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0a665-de58-4485-83b6-2ae178018a0e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:55.596475 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0a665-de58-4485-83b6-2ae178018a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:55.596744 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0a665-de58-4485-83b6-2ae178018a0e None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:55.597471 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0a665-de58-4485-83b6-2ae178018a0e None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:55.597710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0a665-de58-4485-83b6-2ae178018a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:55.599163 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0a665-de58-4485-83b6-2ae178018a0e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:55.609566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 64/122] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:24:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:55.610229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0a665-de58-4485-83b6-2ae178018a0e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:55.673382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ce872-f35e-4c08-b5e4-783c1779fa6d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:55.682635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:55.757217 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1906a-72f5-431a-8782-22ab9fccf682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:55.758667 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1906a-72f5-431a-8782-22ab9fccf682 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:55.759431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1906a-72f5-431a-8782-22ab9fccf682 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:55.760151 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1906a-72f5-431a-8782-22ab9fccf682 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:55.760719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0a665-de58-4485-83b6-2ae178018a0e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:55.807935 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:55.855625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5778f-30f5-4da8-81bf-6445673b8e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:55.857106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5778f-30f5-4da8-81bf-6445673b8e23 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:55.857935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5778f-30f5-4da8-81bf-6445673b8e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:55.858719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5778f-30f5-4da8-81bf-6445673b8e23 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:55.860145 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5778f-30f5-4da8-81bf-6445673b8e23 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:55.860981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5778f-30f5-4da8-81bf-6445673b8e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:55.864438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5778f-30f5-4da8-81bf-6445673b8e23 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:55.869711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e1906a-72f5-431a-8782-22ab9fccf682 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:55.871272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e1906a-72f5-431a-8782-22ab9fccf682 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:55.883233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 65/123] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:24:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:55.885218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5778f-30f5-4da8-81bf-6445673b8e23 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:55.989593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 59/124] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:24:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:24:55.990570 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1906a-72f5-431a-8782-22ab9fccf682 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:56.052842 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:56.099986 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5778f-30f5-4da8-81bf-6445673b8e23 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:56.115168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1906a-72f5-431a-8782-22ab9fccf682 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:56.155069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:56.159723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890c8d7-cd3c-4ee6-912d-3b87d4ac6a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:56.161270 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890c8d7-cd3c-4ee6-912d-3b87d4ac6a04 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:56.161703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890c8d7-cd3c-4ee6-912d-3b87d4ac6a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:56.162120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890c8d7-cd3c-4ee6-912d-3b87d4ac6a04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:56.176024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890c8d7-cd3c-4ee6-912d-3b87d4ac6a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:24:56.179586 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c999214-531a-4076-b697-76d77d1c6170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:56.180798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c999214-531a-4076-b697-76d77d1c6170 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:56.181420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c999214-531a-4076-b697-76d77d1c6170 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:56.182018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c999214-531a-4076-b697-76d77d1c6170 None admin] PATH_INFO: `/v3/users/107dc688e53144dea3e37d8737d74742` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2890c8d7-cd3c-4ee6-912d-3b87d4ac6a04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:24:56.183010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:24:56.183912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:24:56.184681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:24:56.184681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:24:56.184681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:24:56.184681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:24:56.184681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:24:56.184681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:24:56.184681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:24:56.184681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:24:56.184681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:24:56.184681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:24:56.184681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:24:56.184681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:24:56.187414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c999214-531a-4076-b697-76d77d1c6170 None admin] RBAC: Authorizing `identity:get_user(user_id=107dc688e53144dea3e37d8737d74742)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:56.187892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 60/125] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:24:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:24:56.189049 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890c8d7-cd3c-4ee6-912d-3b87d4ac6a04 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:56.190717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c999214-531a-4076-b697-76d77d1c6170 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:56.198339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 66/126] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:24:55 2022] GET /identity/v3/users/107dc688e53144dea3e37d8737d74742 => generated 275 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:56.199533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c999214-531a-4076-b697-76d77d1c6170 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:56.344535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c999214-531a-4076-b697-76d77d1c6170 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:56.389892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406c3b3-1c93-40ba-923f-7b361bea211e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:56.390697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406c3b3-1c93-40ba-923f-7b361bea211e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:56.391040 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406c3b3-1c93-40ba-923f-7b361bea211e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:56.391376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406c3b3-1c93-40ba-923f-7b361bea211e None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:56.392690 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406c3b3-1c93-40ba-923f-7b361bea211e None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:56.394719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406c3b3-1c93-40ba-923f-7b361bea211e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:56.402183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 67/127] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:24:56 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:56.402183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406c3b3-1c93-40ba-923f-7b361bea211e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:56.447637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890c8d7-cd3c-4ee6-912d-3b87d4ac6a04 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:56.462850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:56.496410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294e7b8-c63a-444f-913f-fa04b68aee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:56.497289 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294e7b8-c63a-444f-913f-fa04b68aee0f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:56.497705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294e7b8-c63a-444f-913f-fa04b68aee0f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:56.498185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294e7b8-c63a-444f-913f-fa04b68aee0f None admin] PATH_INFO: `/v3/users/107dc688e53144dea3e37d8737d74742` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:56.499881 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294e7b8-c63a-444f-913f-fa04b68aee0f None admin] RBAC: Authorizing `identity:get_user(user_id=107dc688e53144dea3e37d8737d74742)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:56.506006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294e7b8-c63a-444f-913f-fa04b68aee0f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:56.506202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:56.506789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 61/128] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:24:56 2022] GET /identity/v3/users/107dc688e53144dea3e37d8737d74742 => generated 275 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:56.506789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294e7b8-c63a-444f-913f-fa04b68aee0f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:56.565631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406c3b3-1c93-40ba-923f-7b361bea211e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:56.624469 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bd2cb-ab0c-4dd8-9069-f6d28a641294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:56.625387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bd2cb-ab0c-4dd8-9069-f6d28a641294 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:56.625823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bd2cb-ab0c-4dd8-9069-f6d28a641294 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:56.627007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bd2cb-ab0c-4dd8-9069-f6d28a641294 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:56.627982 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bd2cb-ab0c-4dd8-9069-f6d28a641294 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:56.628446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bd2cb-ab0c-4dd8-9069-f6d28a641294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:56.631399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bd2cb-ab0c-4dd8-9069-f6d28a641294 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:56.644298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 68/129] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:24:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:56.646604 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bd2cb-ab0c-4dd8-9069-f6d28a641294 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:56.673523 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294e7b8-c63a-444f-913f-fa04b68aee0f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:56.731552 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc718d-2c4b-45a6-ade8-f08b7e83ca71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:56.732490 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc718d-2c4b-45a6-ade8-f08b7e83ca71 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:56.732919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc718d-2c4b-45a6-ade8-f08b7e83ca71 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:56.733335 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc718d-2c4b-45a6-ade8-f08b7e83ca71 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:56.734760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc718d-2c4b-45a6-ade8-f08b7e83ca71 None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:56.736450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc718d-2c4b-45a6-ade8-f08b7e83ca71 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:56.738498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 62/130] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:24:56 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:56.740866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc718d-2c4b-45a6-ade8-f08b7e83ca71 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:56.823362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:56.891832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bd2cb-ab0c-4dd8-9069-f6d28a641294 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:56.925668 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:56.939783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc718d-2c4b-45a6-ade8-f08b7e83ca71 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:56.949633 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b6e17-de9c-49cb-a656-66b392c7527e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:56.950554 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b6e17-de9c-49cb-a656-66b392c7527e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:56.951004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b6e17-de9c-49cb-a656-66b392c7527e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:56.951424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b6e17-de9c-49cb-a656-66b392c7527e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:56.952532 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b6e17-de9c-49cb-a656-66b392c7527e None admin] RBAC: Adding query filter params (role.id=2684d96517574aa7bb00050a3c0e2d92, scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7, user.id=76ae9ead961542bba17e55eac70e16d2) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:24:56.953008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b6e17-de9c-49cb-a656-66b392c7527e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:56.954761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b6e17-de9c-49cb-a656-66b392c7527e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:56.971163 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 69/131] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:24:56 2022] GET /identity/v3/role_assignments?user.id=76ae9ead961542bba17e55eac70e16d2&scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7&role.id=2684d96517574aa7bb00050a3c0e2d92 => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:56.971163 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b6e17-de9c-49cb-a656-66b392c7527e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:57.050427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a11d5-6cfb-4679-9b4f-0bdb595bfdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:57.051524 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a11d5-6cfb-4679-9b4f-0bdb595bfdc4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:57.052003 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a11d5-6cfb-4679-9b4f-0bdb595bfdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:57.052509 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a11d5-6cfb-4679-9b4f-0bdb595bfdc4 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:57.054639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a11d5-6cfb-4679-9b4f-0bdb595bfdc4 None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:57.057305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a11d5-6cfb-4679-9b4f-0bdb595bfdc4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:57.062933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 63/132] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:24:56 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:57.064215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a11d5-6cfb-4679-9b4f-0bdb595bfdc4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:57.245216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a11d5-6cfb-4679-9b4f-0bdb595bfdc4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:57.249618 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b6e17-de9c-49cb-a656-66b392c7527e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:57.281976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:57.310135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f20099-fa1e-46fa-b676-f553c11903ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:57.310519 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3decf4-26f8-4d1e-9949-4c2499dfa093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:57.310839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f20099-fa1e-46fa-b676-f553c11903ae None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:57.311115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f20099-fa1e-46fa-b676-f553c11903ae None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:57.311390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f20099-fa1e-46fa-b676-f553c11903ae None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7/users/107dc688e53144dea3e37d8737d74742/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:57.312014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3decf4-26f8-4d1e-9949-4c2499dfa093 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:57.312622 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3decf4-26f8-4d1e-9949-4c2499dfa093 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:57.313221 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3decf4-26f8-4d1e-9949-4c2499dfa093 None admin] PATH_INFO: `/v3/users/107dc688e53144dea3e37d8737d74742` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:57.313556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f20099-fa1e-46fa-b676-f553c11903ae None admin] RBAC: Authorizing `identity:create_grant(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7, user_id=107dc688e53144dea3e37d8737d74742, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:57.315112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f20099-fa1e-46fa-b676-f553c11903ae None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:57.315697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3decf4-26f8-4d1e-9949-4c2499dfa093 None admin] RBAC: Authorizing `identity:get_user(user_id=107dc688e53144dea3e37d8737d74742)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:57.321321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3decf4-26f8-4d1e-9949-4c2499dfa093 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:57.324528 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 70/133] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:24:56 2022] GET /identity/v3/users/107dc688e53144dea3e37d8737d74742 => generated 275 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:57.325894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3decf4-26f8-4d1e-9949-4c2499dfa093 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:57.428300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 64/134] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:24:57 2022] PUT /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7/users/107dc688e53144dea3e37d8737d74742/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:24:57.428724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:57.431239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6569c02-952e-4bc5-8711-446fa87d683b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:57.431920 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6569c02-952e-4bc5-8711-446fa87d683b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:57.432534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6569c02-952e-4bc5-8711-446fa87d683b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:57.438167 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 65/135] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:24:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:24:57.443615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6569c02-952e-4bc5-8711-446fa87d683b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:57.563574 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3decf4-26f8-4d1e-9949-4c2499dfa093 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:57.661358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740cecb-cf05-4f43-b999-73c7935560ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:57.662052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740cecb-cf05-4f43-b999-73c7935560ad None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:57.662354 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740cecb-cf05-4f43-b999-73c7935560ad None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:57.662626 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740cecb-cf05-4f43-b999-73c7935560ad None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7/users/107dc688e53144dea3e37d8737d74742/roles/65f33fe3045c433b9a3e045e7572aed6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:57.668017 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740cecb-cf05-4f43-b999-73c7935560ad None admin] RBAC: Authorizing `identity:create_grant(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7, user_id=107dc688e53144dea3e37d8737d74742, role_id=65f33fe3045c433b9a3e045e7572aed6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:57.669812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740cecb-cf05-4f43-b999-73c7935560ad None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:57.734699 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6569c02-952e-4bc5-8711-446fa87d683b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:57.777000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 71/136] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:24:57 2022] PUT /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7/users/107dc688e53144dea3e37d8737d74742/roles/65f33fe3045c433b9a3e045e7572aed6 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:24:57.778317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c7287-970a-4626-9633-98c073af3901 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:57.778947 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c7287-970a-4626-9633-98c073af3901 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:57.779228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c7287-970a-4626-9633-98c073af3901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:57.804637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c8f10-66f2-48da-8b33-75185066f520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:57.805979 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c8f10-66f2-48da-8b33-75185066f520 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:57.806517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c8f10-66f2-48da-8b33-75185066f520 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:57.807068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c8f10-66f2-48da-8b33-75185066f520 None admin] PATH_INFO: `/v3/projects/2d2d9cc801e64f22b0116db19cb339f0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:57.808647 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-556c7287-970a-4626-9633-98c073af3901 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:24:57.809234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c8f10-66f2-48da-8b33-75185066f520 None admin] RBAC: Authorizing `identity:get_project(project_id=2d2d9cc801e64f22b0116db19cb339f0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:57.809874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-556c7287-970a-4626-9633-98c073af3901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:24:57.812494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c8f10-66f2-48da-8b33-75185066f520 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:57.815931 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 66/137] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:24:57 2022] GET /identity/v3/projects/2d2d9cc801e64f22b0116db19cb339f0 => generated 312 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:24:57.828771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c8f10-66f2-48da-8b33-75185066f520 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:57.955996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 72/138] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:24:57.985295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:58.162553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c8f10-66f2-48da-8b33-75185066f520 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:58.248930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe8004-fb03-4aa0-afbb-f4c74240f2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:58.249872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe8004-fb03-4aa0-afbb-f4c74240f2fc None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:58.250407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe8004-fb03-4aa0-afbb-f4c74240f2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:58.250820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe8004-fb03-4aa0-afbb-f4c74240f2fc None admin] PATH_INFO: `/v3/projects/2d2d9cc801e64f22b0116db19cb339f0/users/107dc688e53144dea3e37d8737d74742/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:58.253223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe8004-fb03-4aa0-afbb-f4c74240f2fc None admin] RBAC: Authorizing `identity:create_grant(project_id=2d2d9cc801e64f22b0116db19cb339f0, user_id=107dc688e53144dea3e37d8737d74742, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:58.255136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe8004-fb03-4aa0-afbb-f4c74240f2fc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:58.334023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 67/139] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:24:57 2022] PUT /identity/v3/projects/2d2d9cc801e64f22b0116db19cb339f0/users/107dc688e53144dea3e37d8737d74742/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:24:58.564066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c7287-970a-4626-9633-98c073af3901 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:58.623012 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:58.705254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c7287-970a-4626-9633-98c073af3901 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:58.751778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ca1dd-cfd9-4de6-9824-efaf864d70a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:58.753136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ca1dd-cfd9-4de6-9824-efaf864d70a4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:58.753816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ca1dd-cfd9-4de6-9824-efaf864d70a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:58.754501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ca1dd-cfd9-4de6-9824-efaf864d70a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:58.759337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ca1dd-cfd9-4de6-9824-efaf864d70a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:58.762393 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ca1dd-cfd9-4de6-9824-efaf864d70a4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:58.766034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 73/140] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:24:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:24:58.774447 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe8004-fb03-4aa0-afbb-f4c74240f2fc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:24:58.912779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe8004-fb03-4aa0-afbb-f4c74240f2fc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:24:58.914365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:24:58.960037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f25f5-88d3-4ffd-9405-44087d2ba706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:24:58.960751 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f25f5-88d3-4ffd-9405-44087d2ba706 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:24:58.961030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f25f5-88d3-4ffd-9405-44087d2ba706 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:24:58.961297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f25f5-88d3-4ffd-9405-44087d2ba706 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:24:58.961961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f25f5-88d3-4ffd-9405-44087d2ba706 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:24:58.963546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f25f5-88d3-4ffd-9405-44087d2ba706 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:24:59.141153 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 68/141] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:24:58 2022] POST /identity/v3/users => generated 283 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:25:00.183681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cc573-440b-4073-9f49-cbd15c59ebba None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:00.184717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cc573-440b-4073-9f49-cbd15c59ebba None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:00.185223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cc573-440b-4073-9f49-cbd15c59ebba None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:00.187377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 74/142] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:00.199203 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836de1b-4887-4f4c-942b-c2f32859232e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:00.199709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836de1b-4887-4f4c-942b-c2f32859232e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:00.199780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836de1b-4887-4f4c-942b-c2f32859232e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:00.234562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4836de1b-4887-4f4c-942b-c2f32859232e None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:00.235785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4836de1b-4887-4f4c-942b-c2f32859232e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:00.325009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 69/143] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:00.543299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101cfa4-1c7f-4c90-ab14-d49e639eb695 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:00.543897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101cfa4-1c7f-4c90-ab14-d49e639eb695 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:00.544379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101cfa4-1c7f-4c90-ab14-d49e639eb695 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:00.547120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 75/144] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:00.556653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f694954-49ae-48bb-beb4-d1a3a27a94e6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:00.557181 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f694954-49ae-48bb-beb4-d1a3a27a94e6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:00.557648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f694954-49ae-48bb-beb4-d1a3a27a94e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:00.600408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9f694954-49ae-48bb-beb4-d1a3a27a94e6 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:00.602539 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9f694954-49ae-48bb-beb4-d1a3a27a94e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:00.756573 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 70/145] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:00.835481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101cfa4-1c7f-4c90-ab14-d49e639eb695 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:01.097481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101cfa4-1c7f-4c90-ab14-d49e639eb695 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:01.130539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:01.180039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a474cd2-3a6d-426a-ae6a-c0b9a7085284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:01.181081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a474cd2-3a6d-426a-ae6a-c0b9a7085284 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:01.181560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a474cd2-3a6d-426a-ae6a-c0b9a7085284 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:01.182024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a474cd2-3a6d-426a-ae6a-c0b9a7085284 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:01.208934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a474cd2-3a6d-426a-ae6a-c0b9a7085284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0a474cd2-3a6d-426a-ae6a-c0b9a7085284 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:01.223177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:01.224097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:01.224857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:01.224857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:01.224857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:01.224857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:01.224857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:01.224857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:01.224857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:01.224857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:01.224857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:01.224857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:01.224857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:01.224857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:01.225439 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 76/146] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:25:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:25:01.232844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f694954-49ae-48bb-beb4-d1a3a27a94e6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:01.276323 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:01.450253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8965df-f401-4a4d-89db-10b0419a7123 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:01.451565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8965df-f401-4a4d-89db-10b0419a7123 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:01.452331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8965df-f401-4a4d-89db-10b0419a7123 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:01.459315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 77/147] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:01 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:01.460617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f694954-49ae-48bb-beb4-d1a3a27a94e6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:01.466134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133e69f-e2b2-4c9f-8a72-1fd635071076 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:01.466967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133e69f-e2b2-4c9f-8a72-1fd635071076 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:01.467686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133e69f-e2b2-4c9f-8a72-1fd635071076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:01.507858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0133e69f-e2b2-4c9f-8a72-1fd635071076 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:01.510256 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0133e69f-e2b2-4c9f-8a72-1fd635071076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:01.542629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6a0c8-5989-41fe-af71-322bc2fbff4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:01.543599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6a0c8-5989-41fe-af71-322bc2fbff4a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:01.544195 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6a0c8-5989-41fe-af71-322bc2fbff4a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:01.544660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6a0c8-5989-41fe-af71-322bc2fbff4a None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:01.545809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6a0c8-5989-41fe-af71-322bc2fbff4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:01.546471 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6a0c8-5989-41fe-af71-322bc2fbff4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:01.549316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6a0c8-5989-41fe-af71-322bc2fbff4a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:01.566233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 71/148] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:01.568546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6a0c8-5989-41fe-af71-322bc2fbff4a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:01.639302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 78/149] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:01.641264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133e69f-e2b2-4c9f-8a72-1fd635071076 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:01.729946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:01.769845 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6a0c8-5989-41fe-af71-322bc2fbff4a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:01.815456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:01.821774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133e69f-e2b2-4c9f-8a72-1fd635071076 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:01.843362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242970c3-95fe-41e3-a487-f6d4c7e2c76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:01.844767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242970c3-95fe-41e3-a487-f6d4c7e2c76d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:01.845435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242970c3-95fe-41e3-a487-f6d4c7e2c76d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:01.846075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242970c3-95fe-41e3-a487-f6d4c7e2c76d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:01.868257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242970c3-95fe-41e3-a487-f6d4c7e2c76d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-242970c3-95fe-41e3-a487-f6d4c7e2c76d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:01.878834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:01.880453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:01.881927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:01.881927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:01.881927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:01.881927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:01.881927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:01.881927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:01.881927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:01.881927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:01.881927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:01.881927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:01.881927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:01.881927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:01.885986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 72/150] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:01.894871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242970c3-95fe-41e3-a487-f6d4c7e2c76d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:01.939612 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc60a-f11f-4534-8878-f292b502a178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:01.941660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc60a-f11f-4534-8878-f292b502a178 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:01.942139 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc60a-f11f-4534-8878-f292b502a178 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:01.942639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdc60a-f11f-4534-8878-f292b502a178 None admin] PATH_INFO: `/v3/users/e0cb001df8d24a87b7f9253f68745c35` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:01.947857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdc60a-f11f-4534-8878-f292b502a178 None admin] RBAC: Authorizing `identity:get_user(user_id=e0cb001df8d24a87b7f9253f68745c35)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:01.950776 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdc60a-f11f-4534-8878-f292b502a178 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:01.955212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 79/151] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:01 2022] GET /identity/v3/users/e0cb001df8d24a87b7f9253f68745c35 => generated 247 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:01.961719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc60a-f11f-4534-8878-f292b502a178 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:02.180447 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242970c3-95fe-41e3-a487-f6d4c7e2c76d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:02.199261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdc60a-f11f-4534-8878-f292b502a178 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:02.248129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2eb509-6e81-44e6-adf9-dd22764c0f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:02.248990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2eb509-6e81-44e6-adf9-dd22764c0f06 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:02.249138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2eb509-6e81-44e6-adf9-dd22764c0f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:02.249301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2eb509-6e81-44e6-adf9-dd22764c0f06 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:02.249976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2eb509-6e81-44e6-adf9-dd22764c0f06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:02.250315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2eb509-6e81-44e6-adf9-dd22764c0f06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:02.251801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2eb509-6e81-44e6-adf9-dd22764c0f06 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:02.271248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 73/152] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:02.272999 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f903e-ead4-481f-b635-d3cf50c5c7e4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:02.272999 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f903e-ead4-481f-b635-d3cf50c5c7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:02.273326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f903e-ead4-481f-b635-d3cf50c5c7e4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:02.274760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 74/153] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:02.279930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f8840-8f94-47e3-8232-a55e801b41ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:02.280929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3703d2a-8256-46ca-92b0-c5c035066a6b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:02.280929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3703d2a-8256-46ca-92b0-c5c035066a6b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:02.281212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3703d2a-8256-46ca-92b0-c5c035066a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:02.282471 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f8840-8f94-47e3-8232-a55e801b41ce None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:02.283116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f8840-8f94-47e3-8232-a55e801b41ce None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:02.283764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f8840-8f94-47e3-8232-a55e801b41ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:02.306660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:02.318864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e3703d2a-8256-46ca-92b0-c5c035066a6b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:02.320105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e3703d2a-8256-46ca-92b0-c5c035066a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:02.375787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:02.456007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 75/154] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:02.456826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3703d2a-8256-46ca-92b0-c5c035066a6b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:02.471796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f8840-8f94-47e3-8232-a55e801b41ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:02.479536 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f8840-8f94-47e3-8232-a55e801b41ce None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:02.655710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3703d2a-8256-46ca-92b0-c5c035066a6b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:02.718293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:02.739001 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f49d2-e6e8-4249-9c8b-3d36873cb4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:02.739670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f49d2-e6e8-4249-9c8b-3d36873cb4de None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:02.739949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f49d2-e6e8-4249-9c8b-3d36873cb4de None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:02.740217 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f49d2-e6e8-4249-9c8b-3d36873cb4de None admin] PATH_INFO: `/v3/users/76ae9ead961542bba17e55eac70e16d2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:02.741465 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f49d2-e6e8-4249-9c8b-3d36873cb4de None admin] RBAC: Authorizing `identity:get_user(user_id=76ae9ead961542bba17e55eac70e16d2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:02.742983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f49d2-e6e8-4249-9c8b-3d36873cb4de None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:02.744979 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 76/155] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:02 2022] GET /identity/v3/users/76ae9ead961542bba17e55eac70e16d2 => generated 289 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:02.745809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f49d2-e6e8-4249-9c8b-3d36873cb4de None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:02.762463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 80/156] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 802 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:02.763409 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f8840-8f94-47e3-8232-a55e801b41ce None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:02.952691 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f49d2-e6e8-4249-9c8b-3d36873cb4de None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:02.964873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:03.052546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfca269-7c98-4a7b-9a30-c86fef9d0210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:03.053209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfca269-7c98-4a7b-9a30-c86fef9d0210 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:03.053491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfca269-7c98-4a7b-9a30-c86fef9d0210 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:03.053769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfca269-7c98-4a7b-9a30-c86fef9d0210 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:03.077483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfca269-7c98-4a7b-9a30-c86fef9d0210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9cfca269-7c98-4a7b-9a30-c86fef9d0210 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:03.083099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:03.085222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 77/157] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:03.087226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfca269-7c98-4a7b-9a30-c86fef9d0210 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:03.103793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f8840-8f94-47e3-8232-a55e801b41ce None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:03.200220 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:03.231349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9fdd0-d4ba-452c-a065-a5cd650e43af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:03.232526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9fdd0-d4ba-452c-a065-a5cd650e43af None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:03.233023 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9fdd0-d4ba-452c-a065-a5cd650e43af None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:03.233511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9fdd0-d4ba-452c-a065-a5cd650e43af None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:03.235565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9fdd0-d4ba-452c-a065-a5cd650e43af None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:03.238387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee9fdd0-d4ba-452c-a065-a5cd650e43af None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:03.246285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 81/158] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:02 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:03.247896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9fdd0-d4ba-452c-a065-a5cd650e43af None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:03.266265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfca269-7c98-4a7b-9a30-c86fef9d0210 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:03.348640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebfaaf-6419-4a17-975e-185bf0957957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:03.349213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebfaaf-6419-4a17-975e-185bf0957957 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:03.349492 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebfaaf-6419-4a17-975e-185bf0957957 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:03.349762 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebfaaf-6419-4a17-975e-185bf0957957 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:03.354050 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebfaaf-6419-4a17-975e-185bf0957957 None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:03.355842 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebfaaf-6419-4a17-975e-185bf0957957 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:03.357766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 78/159] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:03 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:03.358756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebfaaf-6419-4a17-975e-185bf0957957 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:03.451695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9fdd0-d4ba-452c-a065-a5cd650e43af None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:03.516607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6a024-0595-4ef9-a2e6-271bed2b91b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:03.517299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6a024-0595-4ef9-a2e6-271bed2b91b3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:03.517580 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6a024-0595-4ef9-a2e6-271bed2b91b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:03.517849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6a024-0595-4ef9-a2e6-271bed2b91b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:03.522180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6a024-0595-4ef9-a2e6-271bed2b91b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:03.522180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6a024-0595-4ef9-a2e6-271bed2b91b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:03.522180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6a024-0595-4ef9-a2e6-271bed2b91b3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:03.523493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:03.533502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 82/160] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:03.538175 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6a024-0595-4ef9-a2e6-271bed2b91b3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:03.543069 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebfaaf-6419-4a17-975e-185bf0957957 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:03.613128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:03.629142 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145d94e-9b11-42e8-9576-12bc5b414119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:03.630463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145d94e-9b11-42e8-9576-12bc5b414119 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:03.630927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145d94e-9b11-42e8-9576-12bc5b414119 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:03.631395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145d94e-9b11-42e8-9576-12bc5b414119 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7/users/76ae9ead961542bba17e55eac70e16d2/roles/2684d96517574aa7bb00050a3c0e2d92` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:03.636464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145d94e-9b11-42e8-9576-12bc5b414119 None admin] RBAC: Authorizing `identity:create_grant(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7, user_id=76ae9ead961542bba17e55eac70e16d2, role_id=2684d96517574aa7bb00050a3c0e2d92)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:03.639605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145d94e-9b11-42e8-9576-12bc5b414119 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:03.742386 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 79/161] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:25:03 2022] PUT /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7/users/76ae9ead961542bba17e55eac70e16d2/roles/2684d96517574aa7bb00050a3c0e2d92 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:25:03.745486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145d94e-9b11-42e8-9576-12bc5b414119 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:03.807817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6a024-0595-4ef9-a2e6-271bed2b91b3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:03.923464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfc022-7879-49e8-8e2f-53971f549dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:03.924466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfc022-7879-49e8-8e2f-53971f549dea None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:03.924968 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfc022-7879-49e8-8e2f-53971f549dea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:03.925482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfc022-7879-49e8-8e2f-53971f549dea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:03.926515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfc022-7879-49e8-8e2f-53971f549dea None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7, user.id=e0cb001df8d24a87b7f9253f68745c35) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:03.927034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfc022-7879-49e8-8e2f-53971f549dea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:03.928694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfc022-7879-49e8-8e2f-53971f549dea None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:03.946378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 83/162] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:25:03 2022] GET /identity/v3/role_assignments?user.id=e0cb001df8d24a87b7f9253f68745c35&scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7&role.id=020bc02dca71485bab2312442adaa179 => generated 608 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:03.947538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfc022-7879-49e8-8e2f-53971f549dea None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:04.146320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145d94e-9b11-42e8-9576-12bc5b414119 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:04.152547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:04.226967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfc022-7879-49e8-8e2f-53971f549dea None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:04.251226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73ac02-7d85-4003-9486-b53e030730ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:04.251895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73ac02-7d85-4003-9486-b53e030730ec None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:04.252166 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73ac02-7d85-4003-9486-b53e030730ec None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:04.252435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73ac02-7d85-4003-9486-b53e030730ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:04.279987 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73ac02-7d85-4003-9486-b53e030730ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ec73ac02-7d85-4003-9486-b53e030730ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:04.290190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:04.291124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:04.291905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:04.291905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:04.291905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:04.291905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:04.291905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:04.291905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:04.291905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:04.291905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:04.291905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:04.291905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:04.291905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:04.291905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:04.291905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 80/163] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:25:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 548 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:04.292539 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb450cf-7fd9-4ab4-afe4-fb0778758371 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:04.293102 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb450cf-7fd9-4ab4-afe4-fb0778758371 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:04.293102 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb450cf-7fd9-4ab4-afe4-fb0778758371 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:04.294579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 81/164] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:04.309737 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb450cf-7fd9-4ab4-afe4-fb0778758371 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:04.310810 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1f0d8-ed9c-45b1-aaf3-806bada97f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:04.311468 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1f0d8-ed9c-45b1-aaf3-806bada97f24 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:04.311741 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1f0d8-ed9c-45b1-aaf3-806bada97f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:04.312009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1f0d8-ed9c-45b1-aaf3-806bada97f24 None admin] PATH_INFO: `/v3/users/107dc688e53144dea3e37d8737d74742` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:04.313571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1f0d8-ed9c-45b1-aaf3-806bada97f24 None admin] RBAC: Authorizing `identity:get_user(user_id=107dc688e53144dea3e37d8737d74742)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:04.315146 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1f0d8-ed9c-45b1-aaf3-806bada97f24 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:04.317540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 84/165] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:03 2022] GET /identity/v3/users/107dc688e53144dea3e37d8737d74742 => generated 275 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:04.318623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480e475-8d97-4ff5-aeeb-1a1ac6a6c79f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:04.318898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480e475-8d97-4ff5-aeeb-1a1ac6a6c79f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:04.319165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480e475-8d97-4ff5-aeeb-1a1ac6a6c79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:04.363556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d480e475-8d97-4ff5-aeeb-1a1ac6a6c79f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:04.365119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d480e475-8d97-4ff5-aeeb-1a1ac6a6c79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:04.454967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:04.479650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 85/166] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:04.481224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480e475-8d97-4ff5-aeeb-1a1ac6a6c79f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:04.586175 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:04.645243 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480e475-8d97-4ff5-aeeb-1a1ac6a6c79f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:04.654806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb450cf-7fd9-4ab4-afe4-fb0778758371 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:04.711184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9358e1f-a7e5-403d-bd22-f851bbf02cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:04.712077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9358e1f-a7e5-403d-bd22-f851bbf02cae None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:04.712510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9358e1f-a7e5-403d-bd22-f851bbf02cae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:04.712928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9358e1f-a7e5-403d-bd22-f851bbf02cae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:04.757629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71508044-d799-4178-a798-465023ad81e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:04.758946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71508044-d799-4178-a798-465023ad81e9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:04.759559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71508044-d799-4178-a798-465023ad81e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:04.760257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71508044-d799-4178-a798-465023ad81e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:04.761225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71508044-d799-4178-a798-465023ad81e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:04.761755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71508044-d799-4178-a798-465023ad81e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:04.763501 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71508044-d799-4178-a798-465023ad81e9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:04.782000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 82/167] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:25:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 474 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:04.791556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71508044-d799-4178-a798-465023ad81e9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:04.819609 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9358e1f-a7e5-403d-bd22-f851bbf02cae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:04.821129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9358e1f-a7e5-403d-bd22-f851bbf02cae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:04.916452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:04.976923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 86/168] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:04.984564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9358e1f-a7e5-403d-bd22-f851bbf02cae None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:05.001678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71508044-d799-4178-a798-465023ad81e9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:05.026519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:05.070284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b680b-069b-49a9-9a96-85367f324b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:05.071301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b680b-069b-49a9-9a96-85367f324b6d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:05.071765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b680b-069b-49a9-9a96-85367f324b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:05.072228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b680b-069b-49a9-9a96-85367f324b6d None admin] PATH_INFO: `/v3/users/107dc688e53144dea3e37d8737d74742` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:05.073603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b680b-069b-49a9-9a96-85367f324b6d None admin] RBAC: Authorizing `identity:get_user(user_id=107dc688e53144dea3e37d8737d74742)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:05.075259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b680b-069b-49a9-9a96-85367f324b6d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:05.077423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 83/169] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:04 2022] GET /identity/v3/users/107dc688e53144dea3e37d8737d74742 => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:05.083948 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b680b-069b-49a9-9a96-85367f324b6d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:05.146757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9358e1f-a7e5-403d-bd22-f851bbf02cae None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:05.228372 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac916fa7-53e3-4b24-9e13-e69381386ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:05.229313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac916fa7-53e3-4b24-9e13-e69381386ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:05.229763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac916fa7-53e3-4b24-9e13-e69381386ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:05.230213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac916fa7-53e3-4b24-9e13-e69381386ef5 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:05.232058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac916fa7-53e3-4b24-9e13-e69381386ef5 None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:05.233784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac916fa7-53e3-4b24-9e13-e69381386ef5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:05.241717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 87/170] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:04 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:05.248598 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac916fa7-53e3-4b24-9e13-e69381386ef5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:05.306560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:05.336578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b680b-069b-49a9-9a96-85367f324b6d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:05.417465 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539efaf0-5fdd-4d97-9a9a-2aa2b658fde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:05.418125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539efaf0-5fdd-4d97-9a9a-2aa2b658fde4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:05.418431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539efaf0-5fdd-4d97-9a9a-2aa2b658fde4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:05.418705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539efaf0-5fdd-4d97-9a9a-2aa2b658fde4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:05.479881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac916fa7-53e3-4b24-9e13-e69381386ef5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:05.512349 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:05.573847 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539efaf0-5fdd-4d97-9a9a-2aa2b658fde4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:05.579444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539efaf0-5fdd-4d97-9a9a-2aa2b658fde4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:05.581016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0efa3-6282-4e7c-9303-b27fcd652505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:05.582445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0efa3-6282-4e7c-9303-b27fcd652505 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:05.583248 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0efa3-6282-4e7c-9303-b27fcd652505 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:05.587236 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0efa3-6282-4e7c-9303-b27fcd652505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:05.589236 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0efa3-6282-4e7c-9303-b27fcd652505 None admin] RBAC: Adding query filter params (role.id=dba75ce7d709483fbbb0027d29e99725, scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7, user.id=107dc688e53144dea3e37d8737d74742) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:05.590064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0efa3-6282-4e7c-9303-b27fcd652505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:05.593573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0efa3-6282-4e7c-9303-b27fcd652505 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:05.614280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 88/171] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:25:05 2022] GET /identity/v3/role_assignments?user.id=107dc688e53144dea3e37d8737d74742&scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7&role.id=dba75ce7d709483fbbb0027d29e99725 => generated 608 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:05.615658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0efa3-6282-4e7c-9303-b27fcd652505 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:05.818546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 84/172] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 735 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 08:25:05.820510 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a633c5-af23-42e7-a86e-a9395b0e4660 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:05.821048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a633c5-af23-42e7-a86e-a9395b0e4660 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:05.821631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a633c5-af23-42e7-a86e-a9395b0e4660 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:05.826538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:05.828542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 85/173] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:05.829773 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a633c5-af23-42e7-a86e-a9395b0e4660 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:05.841086 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0efa3-6282-4e7c-9303-b27fcd652505 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:05.916737 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd6521-8144-4888-9a56-71c1d16f5d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:05.917427 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd6521-8144-4888-9a56-71c1d16f5d24 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:05.917875 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd6521-8144-4888-9a56-71c1d16f5d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:05.918169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd6521-8144-4888-9a56-71c1d16f5d24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:05.946356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd6521-8144-4888-9a56-71c1d16f5d24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1cdd6521-8144-4888-9a56-71c1d16f5d24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:05.952193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:05.953141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:05.953913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:05.953913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:05.953913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:05.953913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:05.953913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:05.953913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:05.953913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:05.953913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:05.953913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:05.953913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:05.953913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:05.953913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:05.953913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 89/174] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:05.961822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de212481-a052-4d00-9ddc-c38951e70221 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:05.962179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de212481-a052-4d00-9ddc-c38951e70221 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:05.962407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de212481-a052-4d00-9ddc-c38951e70221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:05.996378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-de212481-a052-4d00-9ddc-c38951e70221 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:05.997617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-de212481-a052-4d00-9ddc-c38951e70221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:06.036166 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:06.070301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a633c5-af23-42e7-a86e-a9395b0e4660 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:06.122753 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c237414-7468-4551-b553-2fe32e6799e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:06.124687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c237414-7468-4551-b553-2fe32e6799e5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:06.125225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c237414-7468-4551-b553-2fe32e6799e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:06.125943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c237414-7468-4551-b553-2fe32e6799e5 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:06.127621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c237414-7468-4551-b553-2fe32e6799e5 None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:06.130001 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c237414-7468-4551-b553-2fe32e6799e5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:06.130128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 90/175] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:06.131179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de212481-a052-4d00-9ddc-c38951e70221 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:06.132510 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 86/176] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:05 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:06.134132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4047dd-ff0f-4d2b-ad0f-0846472fa4df None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:06.134651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4047dd-ff0f-4d2b-ad0f-0846472fa4df None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:06.135093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4047dd-ff0f-4d2b-ad0f-0846472fa4df None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:06.138462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 87/177] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:06.139618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4047dd-ff0f-4d2b-ad0f-0846472fa4df None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:06.238882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:06.263731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de212481-a052-4d00-9ddc-c38951e70221 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:06.293615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4047dd-ff0f-4d2b-ad0f-0846472fa4df None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:06.319095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2140df-8c61-4680-b1ba-1e9143270627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:06.320026 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2140df-8c61-4680-b1ba-1e9143270627 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:06.320876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2140df-8c61-4680-b1ba-1e9143270627 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:06.321397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2140df-8c61-4680-b1ba-1e9143270627 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:06.322800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2140df-8c61-4680-b1ba-1e9143270627 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:06.323363 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2140df-8c61-4680-b1ba-1e9143270627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:06.324979 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2140df-8c61-4680-b1ba-1e9143270627 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:06.337626 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 91/178] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:06.340104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bc5d2-7dfc-4937-a16f-690189350e89 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:06.340654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bc5d2-7dfc-4937-a16f-690189350e89 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:06.341261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bc5d2-7dfc-4937-a16f-690189350e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:06.348596 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf66dd4-4665-4973-9264-6620e6f1d236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:06.349205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf66dd4-4665-4973-9264-6620e6f1d236 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:06.349474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf66dd4-4665-4973-9264-6620e6f1d236 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:06.349744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf66dd4-4665-4973-9264-6620e6f1d236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:06.350526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf66dd4-4665-4973-9264-6620e6f1d236 None admin] RBAC: Adding query filter params (role.id=65f33fe3045c433b9a3e045e7572aed6, scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7, user.id=107dc688e53144dea3e37d8737d74742) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:06.350829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf66dd4-4665-4973-9264-6620e6f1d236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:06.352300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf66dd4-4665-4973-9264-6620e6f1d236 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:06.363150 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 88/179] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:25:06 2022] GET /identity/v3/role_assignments?user.id=107dc688e53144dea3e37d8737d74742&scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7&role.id=65f33fe3045c433b9a3e045e7572aed6 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:06.365041 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ca728-320a-4964-8cc4-b0c8017757f6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:06.365320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ca728-320a-4964-8cc4-b0c8017757f6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:06.365581 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ca728-320a-4964-8cc4-b0c8017757f6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:06.370237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 89/180] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:06.370237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ca728-320a-4964-8cc4-b0c8017757f6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:06.385644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:06.395418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-281bc5d2-7dfc-4937-a16f-690189350e89 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:06.396690 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-281bc5d2-7dfc-4937-a16f-690189350e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:06.539237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 92/181] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:06.546966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778669-41fc-4f76-9eb6-8bc3a0c407d4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:06.547269 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778669-41fc-4f76-9eb6-8bc3a0c407d4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:06.547518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778669-41fc-4f76-9eb6-8bc3a0c407d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:06.580104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4e778669-41fc-4f76-9eb6-8bc3a0c407d4 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:06.581315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4e778669-41fc-4f76-9eb6-8bc3a0c407d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:06.608340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ca728-320a-4964-8cc4-b0c8017757f6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:06.711761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5265f-d800-4feb-8eae-566bfb3627f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:06.713326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5265f-d800-4feb-8eae-566bfb3627f8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:06.714599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5265f-d800-4feb-8eae-566bfb3627f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:06.716730 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5265f-d800-4feb-8eae-566bfb3627f8 None admin] PATH_INFO: `/v3/users/107dc688e53144dea3e37d8737d74742` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:06.719458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5265f-d800-4feb-8eae-566bfb3627f8 None admin] RBAC: Authorizing `identity:get_user(user_id=107dc688e53144dea3e37d8737d74742)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:06.720685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 93/182] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:06.721587 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e778669-41fc-4f76-9eb6-8bc3a0c407d4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:06.724084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5265f-d800-4feb-8eae-566bfb3627f8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:06.725965 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:06.733595 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 90/183] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:06 2022] GET /identity/v3/users/107dc688e53144dea3e37d8737d74742 => generated 275 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:06.743707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5265f-d800-4feb-8eae-566bfb3627f8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:06.929171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e778669-41fc-4f76-9eb6-8bc3a0c407d4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:06.956366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:07.000143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5265f-d800-4feb-8eae-566bfb3627f8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:07.010268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dcd4a8-be41-4dd9-84d6-99988bfe4652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:07.010268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcd4a8-be41-4dd9-84d6-99988bfe4652 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:07.010268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcd4a8-be41-4dd9-84d6-99988bfe4652 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:07.010268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcd4a8-be41-4dd9-84d6-99988bfe4652 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:07.040105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dcd4a8-be41-4dd9-84d6-99988bfe4652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-82dcd4a8-be41-4dd9-84d6-99988bfe4652 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:07.050583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:07.052374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:07.055092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:07.055092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:07.055092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:07.055092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:07.055092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:07.055092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:07.055092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:07.055092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:07.055092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:07.055092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:07.055092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:07.055092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:07.057491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 94/184] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:25:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:07.057959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dcd4a8-be41-4dd9-84d6-99988bfe4652 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:07.083091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905117bd-ce3e-4938-bf25-c3ed5463e2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:07.085275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905117bd-ce3e-4938-bf25-c3ed5463e2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:07.086620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905117bd-ce3e-4938-bf25-c3ed5463e2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:07.087419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905117bd-ce3e-4938-bf25-c3ed5463e2c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:07.187390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:07.233441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905117bd-ce3e-4938-bf25-c3ed5463e2c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:07.235535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905117bd-ce3e-4938-bf25-c3ed5463e2c0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:07.244903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dcd4a8-be41-4dd9-84d6-99988bfe4652 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:07.296196 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2569f-61f4-4457-9f3a-7c43c0f9d815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:07.296899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2569f-61f4-4457-9f3a-7c43c0f9d815 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:07.297175 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2569f-61f4-4457-9f3a-7c43c0f9d815 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:07.297445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2569f-61f4-4457-9f3a-7c43c0f9d815 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:07.308552 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2569f-61f4-4457-9f3a-7c43c0f9d815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-48a2569f-61f4-4457-9f3a-7c43c0f9d815 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:07.313565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:07.314509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 95/185] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:25:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:07.315615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2569f-61f4-4457-9f3a-7c43c0f9d815 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:07.337078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:07.364122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 91/186] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:07.364887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905117bd-ce3e-4938-bf25-c3ed5463e2c0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:07.431825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2569f-61f4-4457-9f3a-7c43c0f9d815 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:07.456578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:07.468147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905117bd-ce3e-4938-bf25-c3ed5463e2c0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:07.468573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce059057-f69f-4a9b-bca6-c94fdd374d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:07.469105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce059057-f69f-4a9b-bca6-c94fdd374d33 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:07.469378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce059057-f69f-4a9b-bca6-c94fdd374d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:07.469774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce059057-f69f-4a9b-bca6-c94fdd374d33 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:07.470328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce059057-f69f-4a9b-bca6-c94fdd374d33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:07.470634 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce059057-f69f-4a9b-bca6-c94fdd374d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:07.472019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce059057-f69f-4a9b-bca6-c94fdd374d33 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:07.480904 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 96/187] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:07.482359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbece1f1-d96e-4d50-883d-fcf70328a05a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:07.482643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbece1f1-d96e-4d50-883d-fcf70328a05a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:07.482908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbece1f1-d96e-4d50-883d-fcf70328a05a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:07.485227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 97/188] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:07.485227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbece1f1-d96e-4d50-883d-fcf70328a05a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:07.526823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2447d8a-09be-4168-a8d2-6cc522d2bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:07.527496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2447d8a-09be-4168-a8d2-6cc522d2bc38 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:07.527777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2447d8a-09be-4168-a8d2-6cc522d2bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:07.528336 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2447d8a-09be-4168-a8d2-6cc522d2bc38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:07.529616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2447d8a-09be-4168-a8d2-6cc522d2bc38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:07.531178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2447d8a-09be-4168-a8d2-6cc522d2bc38 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:07.533002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 92/189] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:25:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:07.535754 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2447d8a-09be-4168-a8d2-6cc522d2bc38 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:07.590954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:07.611805 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbece1f1-d96e-4d50-883d-fcf70328a05a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:07.658101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2447d8a-09be-4168-a8d2-6cc522d2bc38 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:07.668791 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419815b6-0333-4466-a760-80f09a4ffcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:07.669404 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419815b6-0333-4466-a760-80f09a4ffcf0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:07.669675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419815b6-0333-4466-a760-80f09a4ffcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:07.669942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419815b6-0333-4466-a760-80f09a4ffcf0 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:07.670649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419815b6-0333-4466-a760-80f09a4ffcf0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:07.670945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419815b6-0333-4466-a760-80f09a4ffcf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:07.672359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419815b6-0333-4466-a760-80f09a4ffcf0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:07.682220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 98/190] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:25:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:07.683101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419815b6-0333-4466-a760-80f09a4ffcf0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:07.702456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d311e-843a-4d66-b1fe-3605e955ae78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:07.703300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d311e-843a-4d66-b1fe-3605e955ae78 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:07.703671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d311e-843a-4d66-b1fe-3605e955ae78 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:07.703998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d311e-843a-4d66-b1fe-3605e955ae78 None admin] PATH_INFO: `/v3/projects/2d2d9cc801e64f22b0116db19cb339f0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:07.705241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d311e-843a-4d66-b1fe-3605e955ae78 None admin] RBAC: Authorizing `identity:get_project(project_id=2d2d9cc801e64f22b0116db19cb339f0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:07.707084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d311e-843a-4d66-b1fe-3605e955ae78 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:07.709085 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 93/191] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:07 2022] GET /identity/v3/projects/2d2d9cc801e64f22b0116db19cb339f0 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:07.715853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3a824-2d58-4a32-a319-b195fe310b4a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:07.715853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3a824-2d58-4a32-a319-b195fe310b4a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:07.715853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3a824-2d58-4a32-a319-b195fe310b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:07.730182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f8a3a824-2d58-4a32-a319-b195fe310b4a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:07.731410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f8a3a824-2d58-4a32-a319-b195fe310b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:07.751290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:07.810268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419815b6-0333-4466-a760-80f09a4ffcf0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:07.819205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 94/192] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:07.820112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3a824-2d58-4a32-a319-b195fe310b4a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:07.877566 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:07.879601 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a5998-6c7a-4f77-837c-1e55c6a7028c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:07.881034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a5998-6c7a-4f77-837c-1e55c6a7028c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:07.881530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a5998-6c7a-4f77-837c-1e55c6a7028c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:07.881992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a5998-6c7a-4f77-837c-1e55c6a7028c None admin] PATH_INFO: `/v3/users/3691aa3013b84bc29d308fafe6c9b222` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:07.912776 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a5998-6c7a-4f77-837c-1e55c6a7028c None admin] RBAC: Authorizing `identity:get_user(user_id=3691aa3013b84bc29d308fafe6c9b222)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:07.915546 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a5998-6c7a-4f77-837c-1e55c6a7028c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:07.919476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 99/193] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:07 2022] GET /identity/v3/users/3691aa3013b84bc29d308fafe6c9b222 => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:07.920723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a5998-6c7a-4f77-837c-1e55c6a7028c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:07.953664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3a824-2d58-4a32-a319-b195fe310b4a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:07.995000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d93cf-ea3c-4f81-bf30-5616ea2b0cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:07.995497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d93cf-ea3c-4f81-bf30-5616ea2b0cba None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:07.995764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d93cf-ea3c-4f81-bf30-5616ea2b0cba None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:07.996022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d93cf-ea3c-4f81-bf30-5616ea2b0cba None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:07.996674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d93cf-ea3c-4f81-bf30-5616ea2b0cba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:07.998129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d93cf-ea3c-4f81-bf30-5616ea2b0cba None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:08.095904 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a5998-6c7a-4f77-837c-1e55c6a7028c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:08.105302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:08.142179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 95/194] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:25:07 2022] POST /identity/v3/users => generated 297 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:25:08.142179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d93cf-ea3c-4f81-bf30-5616ea2b0cba None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:08.143564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ae09f-8279-47e6-bad9-cad74738d554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:08.144734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ae09f-8279-47e6-bad9-cad74738d554 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:08.145239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ae09f-8279-47e6-bad9-cad74738d554 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:08.145733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ae09f-8279-47e6-bad9-cad74738d554 None admin] PATH_INFO: `/v3/users/3691aa3013b84bc29d308fafe6c9b222` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:08.147750 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ae09f-8279-47e6-bad9-cad74738d554 None admin] RBAC: Authorizing `identity:get_user(user_id=3691aa3013b84bc29d308fafe6c9b222)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:08.151103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ae09f-8279-47e6-bad9-cad74738d554 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:08.154658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 100/195] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:07 2022] GET /identity/v3/users/3691aa3013b84bc29d308fafe6c9b222 => generated 283 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:08.155910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ae09f-8279-47e6-bad9-cad74738d554 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:08.307817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:08.313350 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d93cf-ea3c-4f81-bf30-5616ea2b0cba None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:08.317661 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ae09f-8279-47e6-bad9-cad74738d554 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:08.376644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed3b5c-6605-497f-80ab-b2cbe95dbe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:08.377598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed3b5c-6605-497f-80ab-b2cbe95dbe8e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:08.378006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed3b5c-6605-497f-80ab-b2cbe95dbe8e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:08.378455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed3b5c-6605-497f-80ab-b2cbe95dbe8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:08.378701 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825eec9-c0c5-489d-a362-e651a11c4f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:08.379474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825eec9-c0c5-489d-a362-e651a11c4f14 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:08.379724 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed3b5c-6605-497f-80ab-b2cbe95dbe8e None admin] RBAC: Adding query filter params (role.id=dba75ce7d709483fbbb0027d29e99725, scope.project.id=2d2d9cc801e64f22b0116db19cb339f0, user.id=107dc688e53144dea3e37d8737d74742) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:08.379967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825eec9-c0c5-489d-a362-e651a11c4f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:08.380223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed3b5c-6605-497f-80ab-b2cbe95dbe8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:08.380468 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825eec9-c0c5-489d-a362-e651a11c4f14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:08.382492 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed3b5c-6605-497f-80ab-b2cbe95dbe8e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:08.392115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 96/196] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:25:08 2022] GET /identity/v3/role_assignments?user.id=107dc688e53144dea3e37d8737d74742&scope.project.id=2d2d9cc801e64f22b0116db19cb339f0&role.id=dba75ce7d709483fbbb0027d29e99725 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:08.393526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed3b5c-6605-497f-80ab-b2cbe95dbe8e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:08.462377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:08.518260 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825eec9-c0c5-489d-a362-e651a11c4f14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:08.521826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825eec9-c0c5-489d-a362-e651a11c4f14 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:08.582970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed3b5c-6605-497f-80ab-b2cbe95dbe8e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:08.646669 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbac46-fe2d-4c8b-87e2-5c16809c8c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:08.647587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbac46-fe2d-4c8b-87e2-5c16809c8c21 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:08.648094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbac46-fe2d-4c8b-87e2-5c16809c8c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:08.648572 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbac46-fe2d-4c8b-87e2-5c16809c8c21 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:08.670059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cbac46-fe2d-4c8b-87e2-5c16809c8c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-51cbac46-fe2d-4c8b-87e2-5c16809c8c21 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:08.681473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:08.682967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:08.684256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:08.684256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:08.684256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:08.684256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:08.684256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:08.684256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:08.684256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:08.684256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:08.684256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:08.684256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:08.684256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:08.684256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:08.684256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 97/197] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:08.685302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbac46-fe2d-4c8b-87e2-5c16809c8c21 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:08.747339 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:08.748817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 101/198] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:08.750789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825eec9-c0c5-489d-a362-e651a11c4f14 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:08.920328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbac46-fe2d-4c8b-87e2-5c16809c8c21 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:08.921551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:08.979285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825eec9-c0c5-489d-a362-e651a11c4f14 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:08.991082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfbd9c-7d1c-4f31-af43-6cc7daafa00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:08.992204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfbd9c-7d1c-4f31-af43-6cc7daafa00b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:08.992733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfbd9c-7d1c-4f31-af43-6cc7daafa00b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:08.993254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfbd9c-7d1c-4f31-af43-6cc7daafa00b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:09.056961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae24c8-1f04-420f-aee2-2831359ec9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:09.057975 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae24c8-1f04-420f-aee2-2831359ec9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:09.058469 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae24c8-1f04-420f-aee2-2831359ec9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:09.059247 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae24c8-1f04-420f-aee2-2831359ec9f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:09.060151 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae24c8-1f04-420f-aee2-2831359ec9f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:09.061650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae24c8-1f04-420f-aee2-2831359ec9f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:09.063325 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae24c8-1f04-420f-aee2-2831359ec9f5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:09.074898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 102/199] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:09.076231 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae24c8-1f04-420f-aee2-2831359ec9f5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:09.125469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfbd9c-7d1c-4f31-af43-6cc7daafa00b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:09.127868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfbd9c-7d1c-4f31-af43-6cc7daafa00b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:09.179209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:09.221981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae24c8-1f04-420f-aee2-2831359ec9f5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:09.279779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f87a4-6f6e-41c5-98a5-3d242352fc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:09.281046 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f87a4-6f6e-41c5-98a5-3d242352fc57 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:09.281551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f87a4-6f6e-41c5-98a5-3d242352fc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:09.282043 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f87a4-6f6e-41c5-98a5-3d242352fc57 None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:09.284395 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f87a4-6f6e-41c5-98a5-3d242352fc57 None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:09.287637 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f87a4-6f6e-41c5-98a5-3d242352fc57 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:09.291093 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 103/200] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:09 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:09.292576 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f87a4-6f6e-41c5-98a5-3d242352fc57 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:09.301301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:09.308074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 98/201] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:09.309289 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfbd9c-7d1c-4f31-af43-6cc7daafa00b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:09.473764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfbd9c-7d1c-4f31-af43-6cc7daafa00b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:09.503101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:09.505383 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f87a4-6f6e-41c5-98a5-3d242352fc57 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:09.515559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606d2ae-9b1e-48b8-b42f-872d7a8c79fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:09.516192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606d2ae-9b1e-48b8-b42f-872d7a8c79fd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:09.516578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606d2ae-9b1e-48b8-b42f-872d7a8c79fd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:09.516851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606d2ae-9b1e-48b8-b42f-872d7a8c79fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:09.517589 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606d2ae-9b1e-48b8-b42f-872d7a8c79fd None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=ac74b750ef33436a83f5ee0474a71920, user.id=3691aa3013b84bc29d308fafe6c9b222) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:09.517893 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606d2ae-9b1e-48b8-b42f-872d7a8c79fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:09.519473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606d2ae-9b1e-48b8-b42f-872d7a8c79fd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:09.526301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 99/202] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:25:09 2022] GET /identity/v3/role_assignments?user.id=3691aa3013b84bc29d308fafe6c9b222&scope.project.id=ac74b750ef33436a83f5ee0474a71920&role.id=020bc02dca71485bab2312442adaa179 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:09.527295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606d2ae-9b1e-48b8-b42f-872d7a8c79fd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:09.577496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8d089-e53b-4230-8a89-21d44b61402e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:09.579132 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8d089-e53b-4230-8a89-21d44b61402e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:09.579943 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8d089-e53b-4230-8a89-21d44b61402e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:09.580701 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8d089-e53b-4230-8a89-21d44b61402e None admin] PATH_INFO: `/v3/users/76ae9ead961542bba17e55eac70e16d2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:09.587967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8d089-e53b-4230-8a89-21d44b61402e None admin] RBAC: Authorizing `identity:get_user(user_id=76ae9ead961542bba17e55eac70e16d2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:09.591379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8d089-e53b-4230-8a89-21d44b61402e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:09.599676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 104/203] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:09 2022] GET /identity/v3/users/76ae9ead961542bba17e55eac70e16d2 => generated 289 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:09.609456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8d089-e53b-4230-8a89-21d44b61402e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:09.732697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606d2ae-9b1e-48b8-b42f-872d7a8c79fd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:09.757199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:09.815103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9c917-01f3-41be-8579-60a5934371d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:09.815851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9c917-01f3-41be-8579-60a5934371d7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:09.816133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9c917-01f3-41be-8579-60a5934371d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:09.816451 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9c917-01f3-41be-8579-60a5934371d7 None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:09.817732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9c917-01f3-41be-8579-60a5934371d7 None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:09.819567 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9c917-01f3-41be-8579-60a5934371d7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:09.821444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 100/204] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:09 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:09.821809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8d089-e53b-4230-8a89-21d44b61402e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:09.828697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9c917-01f3-41be-8579-60a5934371d7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:09.974819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec412f-fb80-46fc-9ca0-ea0bb497f6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:09.976455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec412f-fb80-46fc-9ca0-ea0bb497f6ce None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:09.977489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec412f-fb80-46fc-9ca0-ea0bb497f6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:09.978430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec412f-fb80-46fc-9ca0-ea0bb497f6ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:10.188248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:10.267474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec412f-fb80-46fc-9ca0-ea0bb497f6ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:10.271077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec412f-fb80-46fc-9ca0-ea0bb497f6ce None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:10.437937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:10.488486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9c917-01f3-41be-8579-60a5934371d7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:10.506281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 105/205] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 893 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:10.513283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec412f-fb80-46fc-9ca0-ea0bb497f6ce None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:10.670119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790fa7b-8ade-4dfa-93f5-42ff6cbcc442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:10.670119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790fa7b-8ade-4dfa-93f5-42ff6cbcc442 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:10.670119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790fa7b-8ade-4dfa-93f5-42ff6cbcc442 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:10.670119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790fa7b-8ade-4dfa-93f5-42ff6cbcc442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:10.677593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790fa7b-8ade-4dfa-93f5-42ff6cbcc442 None admin] RBAC: Adding query filter params (role.id=65f33fe3045c433b9a3e045e7572aed6, scope.project.id=ac74b750ef33436a83f5ee0474a71920, user.id=3691aa3013b84bc29d308fafe6c9b222) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:10.678046 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790fa7b-8ade-4dfa-93f5-42ff6cbcc442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:10.682192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790fa7b-8ade-4dfa-93f5-42ff6cbcc442 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:10.710199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 101/206] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:25:09 2022] GET /identity/v3/role_assignments?user.id=3691aa3013b84bc29d308fafe6c9b222&scope.project.id=ac74b750ef33436a83f5ee0474a71920&role.id=65f33fe3045c433b9a3e045e7572aed6 => generated 259 bytes in 880 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:10.911755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec412f-fb80-46fc-9ca0-ea0bb497f6ce None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:10.968164 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cdf5e8-cc17-42d4-9896-4e6debd4172c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:10.968718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cdf5e8-cc17-42d4-9896-4e6debd4172c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:10.969038 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cdf5e8-cc17-42d4-9896-4e6debd4172c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:10.969319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cdf5e8-cc17-42d4-9896-4e6debd4172c None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:10.975087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cdf5e8-cc17-42d4-9896-4e6debd4172c None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:10.976611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cdf5e8-cc17-42d4-9896-4e6debd4172c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:10.978460 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 106/207] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:10 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:10.988083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790fa7b-8ade-4dfa-93f5-42ff6cbcc442 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:11.145996 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790fa7b-8ade-4dfa-93f5-42ff6cbcc442 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:11.191155 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:11.223306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4768794-796b-4d8f-88a5-e3ca79ea4b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:11.224305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4768794-796b-4d8f-88a5-e3ca79ea4b13 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:11.224778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4768794-796b-4d8f-88a5-e3ca79ea4b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:11.225211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4768794-796b-4d8f-88a5-e3ca79ea4b13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:11.226574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4768794-796b-4d8f-88a5-e3ca79ea4b13 None admin] RBAC: Adding query filter params (role.id=2684d96517574aa7bb00050a3c0e2d92, scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7, user.id=76ae9ead961542bba17e55eac70e16d2) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:11.227092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4768794-796b-4d8f-88a5-e3ca79ea4b13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:11.229479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4768794-796b-4d8f-88a5-e3ca79ea4b13 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:11.246936 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 102/208] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:25:10 2022] GET /identity/v3/role_assignments?user.id=76ae9ead961542bba17e55eac70e16d2&scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7&role.id=2684d96517574aa7bb00050a3c0e2d92 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:11.272332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15d1d6-f731-4ae8-862a-2df18b139124 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:11.272593 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15d1d6-f731-4ae8-862a-2df18b139124 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:11.272905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15d1d6-f731-4ae8-862a-2df18b139124 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:11.274293 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 107/209] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:25:11.283511 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f940d-8162-4188-847a-dde1fdca41fb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:11.284000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f940d-8162-4188-847a-dde1fdca41fb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:11.284399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f940d-8162-4188-847a-dde1fdca41fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:11.321116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c60f940d-8162-4188-847a-dde1fdca41fb None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:11.322410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c60f940d-8162-4188-847a-dde1fdca41fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:11.444762 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 103/210] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:11.895643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15d1d6-f731-4ae8-862a-2df18b139124 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:11.954187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:12.044131 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15d1d6-f731-4ae8-862a-2df18b139124 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:12.090476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3844ee8-91a3-4659-8c1e-bfe11871c779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:12.091379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3844ee8-91a3-4659-8c1e-bfe11871c779 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:12.091827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3844ee8-91a3-4659-8c1e-bfe11871c779 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:12.092235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3844ee8-91a3-4659-8c1e-bfe11871c779 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:12.105959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3844ee8-91a3-4659-8c1e-bfe11871c779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d3844ee8-91a3-4659-8c1e-bfe11871c779 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:12.111381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:12.112458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:12.113378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:12.113378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:12.113378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:12.113378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:12.113378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:12.113378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:12.113378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:12.113378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:12.113378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:12.113378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:12.113378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:12.113378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:12.115411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 108/211] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:25:12.125297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f940d-8162-4188-847a-dde1fdca41fb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:12.273063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60f940d-8162-4188-847a-dde1fdca41fb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:12.290179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:12.303329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11db86b-9877-48b1-b931-dba15a5c7d48 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:12.303764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11db86b-9877-48b1-b931-dba15a5c7d48 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:12.304048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11db86b-9877-48b1-b931-dba15a5c7d48 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:12.306672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 109/212] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:12.317579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35140239-2db6-4add-ad27-46b5d7a79b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:12.317849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1409441-f404-4779-9849-5b314c4a56b6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:12.318283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1409441-f404-4779-9849-5b314c4a56b6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:12.318460 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1409441-f404-4779-9849-5b314c4a56b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:12.318586 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35140239-2db6-4add-ad27-46b5d7a79b71 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:12.318937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35140239-2db6-4add-ad27-46b5d7a79b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:12.319407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35140239-2db6-4add-ad27-46b5d7a79b71 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:12.320599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35140239-2db6-4add-ad27-46b5d7a79b71 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:12.321132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35140239-2db6-4add-ad27-46b5d7a79b71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:12.324196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35140239-2db6-4add-ad27-46b5d7a79b71 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:12.341738 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e1409441-f404-4779-9849-5b314c4a56b6 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:12.342164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 104/213] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:12.343106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e1409441-f404-4779-9849-5b314c4a56b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:12.354816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35140239-2db6-4add-ad27-46b5d7a79b71 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:12.428783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 110/214] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:12.540868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35140239-2db6-4add-ad27-46b5d7a79b71 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:12.602274 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7602-4d0c-403a-a916-42b7eb575143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:12.603328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7602-4d0c-403a-a916-42b7eb575143 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:12.603762 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7602-4d0c-403a-a916-42b7eb575143 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:12.604182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7602-4d0c-403a-a916-42b7eb575143 None admin] PATH_INFO: `/v3/users/d2491e3f0e3e483284c9d49659ecbaf4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:12.629803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa7602-4d0c-403a-a916-42b7eb575143 None admin] RBAC: Authorizing `identity:get_user(user_id=d2491e3f0e3e483284c9d49659ecbaf4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:12.632567 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa7602-4d0c-403a-a916-42b7eb575143 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:12.635890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 105/215] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:12 2022] GET /identity/v3/users/d2491e3f0e3e483284c9d49659ecbaf4 => generated 297 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:12.644700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1409441-f404-4779-9849-5b314c4a56b6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:12.700670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:12.750488 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7602-4d0c-403a-a916-42b7eb575143 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:12.796138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:12.846027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1409441-f404-4779-9849-5b314c4a56b6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:12.885494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7602-4d0c-403a-a916-42b7eb575143 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:12.916843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0c59c-b14e-44a1-a4fa-6c0d8ffce1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:12.917890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0c59c-b14e-44a1-a4fa-6c0d8ffce1dd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:12.918402 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0c59c-b14e-44a1-a4fa-6c0d8ffce1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:12.918934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0c59c-b14e-44a1-a4fa-6c0d8ffce1dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:12.928729 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565dca5c-c385-4359-b0e9-ebded9f8d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:12.929723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565dca5c-c385-4359-b0e9-ebded9f8d9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:12.930287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565dca5c-c385-4359-b0e9-ebded9f8d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:12.930759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565dca5c-c385-4359-b0e9-ebded9f8d9d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:12.932160 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565dca5c-c385-4359-b0e9-ebded9f8d9d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:12.933892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565dca5c-c385-4359-b0e9-ebded9f8d9d0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:12.940811 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 106/216] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:25:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:12.942268 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565dca5c-c385-4359-b0e9-ebded9f8d9d0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:13.054457 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0c59c-b14e-44a1-a4fa-6c0d8ffce1dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:13.057170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0c59c-b14e-44a1-a4fa-6c0d8ffce1dd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:13.063188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565dca5c-c385-4359-b0e9-ebded9f8d9d0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:13.065541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:13.107077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5d0f5-b479-4567-98c5-af37d7065499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:13.108230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5d0f5-b479-4567-98c5-af37d7065499 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:13.108729 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5d0f5-b479-4567-98c5-af37d7065499 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:13.109235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5d0f5-b479-4567-98c5-af37d7065499 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:13.110171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5d0f5-b479-4567-98c5-af37d7065499 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:13.112024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5d0f5-b479-4567-98c5-af37d7065499 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:13.126388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:13.220167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 111/217] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:13.221505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884af6d-3557-4976-8d33-7efd0aa28ca3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:13.221798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884af6d-3557-4976-8d33-7efd0aa28ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:13.222045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884af6d-3557-4976-8d33-7efd0aa28ca3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:13.223792 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 112/218] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:13.228454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884af6d-3557-4976-8d33-7efd0aa28ca3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:13.242192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 107/219] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:25:12 2022] POST /identity/v3/users => generated 297 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:25:13.242192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a0a34-13ed-4c83-9bc8-9e086d7ba933 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:13.242192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a0a34-13ed-4c83-9bc8-9e086d7ba933 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:13.242192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a0a34-13ed-4c83-9bc8-9e086d7ba933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:13.272206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-069a0a34-13ed-4c83-9bc8-9e086d7ba933 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:13.274309 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-069a0a34-13ed-4c83-9bc8-9e086d7ba933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:13.392391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884af6d-3557-4976-8d33-7efd0aa28ca3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:13.410516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 108/220] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:13.433146 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:13.439331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d914a-6609-43ce-baae-9b857e66fe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:13.439996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d914a-6609-43ce-baae-9b857e66fe1c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:13.440323 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d914a-6609-43ce-baae-9b857e66fe1c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:13.440600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d914a-6609-43ce-baae-9b857e66fe1c None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:13.441875 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d914a-6609-43ce-baae-9b857e66fe1c None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:13.443511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d914a-6609-43ce-baae-9b857e66fe1c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:13.445698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 113/221] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:13 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:13.455863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a0a34-13ed-4c83-9bc8-9e086d7ba933 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:13.590880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:13.606500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a0a34-13ed-4c83-9bc8-9e086d7ba933 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:13.649146 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a550b1a4-a8c3-4ce3-9efe-43fa94de17ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:13.649625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550b1a4-a8c3-4ce3-9efe-43fa94de17ba None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:13.649906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550b1a4-a8c3-4ce3-9efe-43fa94de17ba None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:13.650196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550b1a4-a8c3-4ce3-9efe-43fa94de17ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:13.650943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a550b1a4-a8c3-4ce3-9efe-43fa94de17ba None admin] RBAC: Adding query filter params (role.id=dba75ce7d709483fbbb0027d29e99725, scope.project.id=ac74b750ef33436a83f5ee0474a71920, user.id=d2491e3f0e3e483284c9d49659ecbaf4) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:13.651245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a550b1a4-a8c3-4ce3-9efe-43fa94de17ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:13.652711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a550b1a4-a8c3-4ce3-9efe-43fa94de17ba None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:13.659691 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 109/222] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:25:13 2022] GET /identity/v3/role_assignments?user.id=d2491e3f0e3e483284c9d49659ecbaf4&scope.project.id=ac74b750ef33436a83f5ee0474a71920&role.id=dba75ce7d709483fbbb0027d29e99725 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:13.698245 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d914a-6609-43ce-baae-9b857e66fe1c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:13.879901 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d914a-6609-43ce-baae-9b857e66fe1c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:13.922461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f958f7-36d8-413f-a23a-500e00cc896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:13.923034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f958f7-36d8-413f-a23a-500e00cc896a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:13.923310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f958f7-36d8-413f-a23a-500e00cc896a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:13.923581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f958f7-36d8-413f-a23a-500e00cc896a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:13.935959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f958f7-36d8-413f-a23a-500e00cc896a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d2f958f7-36d8-413f-a23a-500e00cc896a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:13.941000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:13.942493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:13.944022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:13.944022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:13.944022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:13.944022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:13.944022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:13.944022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:13.944022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:13.944022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:13.944022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:13.944022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:13.944022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:13.944022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:13.944022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 114/223] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:25:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:25:13.948744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a550b1a4-a8c3-4ce3-9efe-43fa94de17ba None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:14.139051 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a550b1a4-a8c3-4ce3-9efe-43fa94de17ba None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:14.195108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86794423-8b16-4149-a9cd-2bc639f86d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:14.196154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86794423-8b16-4149-a9cd-2bc639f86d3c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:14.196662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86794423-8b16-4149-a9cd-2bc639f86d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:14.197133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86794423-8b16-4149-a9cd-2bc639f86d3c None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:14.198043 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86794423-8b16-4149-a9cd-2bc639f86d3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:14.198611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86794423-8b16-4149-a9cd-2bc639f86d3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:14.200378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86794423-8b16-4149-a9cd-2bc639f86d3c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:14.216605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 110/224] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:14.221849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f958f7-36d8-413f-a23a-500e00cc896a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:14.307232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:14.357963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f958f7-36d8-413f-a23a-500e00cc896a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:14.426876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116b2c3-5e5f-405c-a799-3e3eca9902d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:14.427547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116b2c3-5e5f-405c-a799-3e3eca9902d6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:14.427825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116b2c3-5e5f-405c-a799-3e3eca9902d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:14.428092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116b2c3-5e5f-405c-a799-3e3eca9902d6 None admin] PATH_INFO: `/v3/users/3691aa3013b84bc29d308fafe6c9b222` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:14.431588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116b2c3-5e5f-405c-a799-3e3eca9902d6 None admin] RBAC: Authorizing `identity:get_user(user_id=3691aa3013b84bc29d308fafe6c9b222)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:14.435953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116b2c3-5e5f-405c-a799-3e3eca9902d6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:14.439061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 115/225] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:14 2022] GET /identity/v3/users/3691aa3013b84bc29d308fafe6c9b222 => generated 283 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:14.446596 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86794423-8b16-4149-a9cd-2bc639f86d3c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:14.483571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:14.595306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86794423-8b16-4149-a9cd-2bc639f86d3c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:14.626008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb587fb-ea51-4ed7-bc12-e23443583d03 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:14.626480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb587fb-ea51-4ed7-bc12-e23443583d03 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:14.626945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb587fb-ea51-4ed7-bc12-e23443583d03 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:14.629288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 116/226] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:25:14.635617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e346c-fcd9-4a0a-85f5-c99af094f704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:14.636505 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e346c-fcd9-4a0a-85f5-c99af094f704 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:14.636956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e346c-fcd9-4a0a-85f5-c99af094f704 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:14.637423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e346c-fcd9-4a0a-85f5-c99af094f704 None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:14.638888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e346c-fcd9-4a0a-85f5-c99af094f704 None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:14.640504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e346c-fcd9-4a0a-85f5-c99af094f704 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:14.642372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 111/227] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:14 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:14.648985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb587fb-ea51-4ed7-bc12-e23443583d03 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:14.651155 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b80f1-926c-4a63-a813-7502550d96f9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:14.651624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b80f1-926c-4a63-a813-7502550d96f9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:14.652073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b80f1-926c-4a63-a813-7502550d96f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:14.673024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4c0b80f1-926c-4a63-a813-7502550d96f9 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:14.674688 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4c0b80f1-926c-4a63-a813-7502550d96f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:14.757745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 112/228] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:14.930538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb587fb-ea51-4ed7-bc12-e23443583d03 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:14.960702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:14.970995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85583c4a-a854-4095-8fd8-41d5f7c6711a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:14.972177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85583c4a-a854-4095-8fd8-41d5f7c6711a None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:14.973272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85583c4a-a854-4095-8fd8-41d5f7c6711a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:14.973706 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85583c4a-a854-4095-8fd8-41d5f7c6711a None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920/users/3691aa3013b84bc29d308fafe6c9b222/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:14.975994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85583c4a-a854-4095-8fd8-41d5f7c6711a None admin] RBAC: Authorizing `identity:create_grant(project_id=ac74b750ef33436a83f5ee0474a71920, user_id=3691aa3013b84bc29d308fafe6c9b222, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:14.977620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85583c4a-a854-4095-8fd8-41d5f7c6711a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:15.084874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 117/229] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:25:14 2022] PUT /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920/users/3691aa3013b84bc29d308fafe6c9b222/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:25:15.343164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b80f1-926c-4a63-a813-7502550d96f9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:15.364442 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:15.525885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b80f1-926c-4a63-a813-7502550d96f9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:15.596960 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047e8a7-9eca-4632-9b34-b839040b8118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:15.598719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047e8a7-9eca-4632-9b34-b839040b8118 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:15.599519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047e8a7-9eca-4632-9b34-b839040b8118 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:15.600272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047e8a7-9eca-4632-9b34-b839040b8118 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:15.623578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047e8a7-9eca-4632-9b34-b839040b8118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6047e8a7-9eca-4632-9b34-b839040b8118 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:15.634946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:15.636767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:15.638417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:15.638417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:15.638417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:15.638417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:15.638417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:15.638417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:15.638417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:15.638417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:15.638417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:15.638417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:15.638417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:15.638417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:15.643126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 113/230] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:25:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:25:15.649950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85583c4a-a854-4095-8fd8-41d5f7c6711a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:15.788596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85583c4a-a854-4095-8fd8-41d5f7c6711a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:15.829172 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147329c-5f7b-47e6-9b60-707aead47ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:15.830213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147329c-5f7b-47e6-9b60-707aead47ecb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:15.830708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147329c-5f7b-47e6-9b60-707aead47ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:15.831174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147329c-5f7b-47e6-9b60-707aead47ecb None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:15.832094 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147329c-5f7b-47e6-9b60-707aead47ecb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:15.832623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147329c-5f7b-47e6-9b60-707aead47ecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:15.834321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147329c-5f7b-47e6-9b60-707aead47ecb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:15.844311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 118/231] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:25:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:15.850427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047e8a7-9eca-4632-9b34-b839040b8118 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:15.968818 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:16.008203 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047e8a7-9eca-4632-9b34-b839040b8118 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:16.056605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb950ae-4e93-410e-ae7c-bd6804f0eea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:16.057254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb950ae-4e93-410e-ae7c-bd6804f0eea1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:16.057531 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb950ae-4e93-410e-ae7c-bd6804f0eea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:16.057801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb950ae-4e93-410e-ae7c-bd6804f0eea1 None admin] PATH_INFO: `/v3/users/3691aa3013b84bc29d308fafe6c9b222` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:16.059014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb950ae-4e93-410e-ae7c-bd6804f0eea1 None admin] RBAC: Authorizing `identity:get_user(user_id=3691aa3013b84bc29d308fafe6c9b222)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:16.060453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb950ae-4e93-410e-ae7c-bd6804f0eea1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:16.062050 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 114/232] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:15 2022] GET /identity/v3/users/3691aa3013b84bc29d308fafe6c9b222 => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:16.068482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147329c-5f7b-47e6-9b60-707aead47ecb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:16.146258 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:16.200932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147329c-5f7b-47e6-9b60-707aead47ecb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:16.237540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61adfb-b7c0-4148-83d9-60f94cfe8d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:16.239154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61adfb-b7c0-4148-83d9-60f94cfe8d2c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:16.239632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61adfb-b7c0-4148-83d9-60f94cfe8d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:16.240101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61adfb-b7c0-4148-83d9-60f94cfe8d2c None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:16.241467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61adfb-b7c0-4148-83d9-60f94cfe8d2c None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:16.243477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61adfb-b7c0-4148-83d9-60f94cfe8d2c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:16.245649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 119/233] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:16 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:16.252054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb950ae-4e93-410e-ae7c-bd6804f0eea1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:16.509135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb950ae-4e93-410e-ae7c-bd6804f0eea1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:16.590317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6a848-31bf-44d4-beb0-b230a7c6206f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:16.591466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6a848-31bf-44d4-beb0-b230a7c6206f None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:16.591963 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6a848-31bf-44d4-beb0-b230a7c6206f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:16.592439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6a848-31bf-44d4-beb0-b230a7c6206f None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920/users/3691aa3013b84bc29d308fafe6c9b222/roles/65f33fe3045c433b9a3e045e7572aed6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:16.601355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6a848-31bf-44d4-beb0-b230a7c6206f None admin] RBAC: Authorizing `identity:create_grant(project_id=ac74b750ef33436a83f5ee0474a71920, user_id=3691aa3013b84bc29d308fafe6c9b222, role_id=65f33fe3045c433b9a3e045e7572aed6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:16.604282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6a848-31bf-44d4-beb0-b230a7c6206f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:16.744425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:16.752784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 115/234] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:25:16 2022] PUT /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920/users/3691aa3013b84bc29d308fafe6c9b222/roles/65f33fe3045c433b9a3e045e7572aed6 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:25:16.899692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cb582-2fec-4a04-84f5-a87fe8a0f653 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:16.900703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cb582-2fec-4a04-84f5-a87fe8a0f653 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:16.901177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cb582-2fec-4a04-84f5-a87fe8a0f653 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:16.903253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 120/235] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:16.912386 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee4528-bd61-47fe-9eae-84c23a2f93bd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:16.912880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee4528-bd61-47fe-9eae-84c23a2f93bd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:16.913368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee4528-bd61-47fe-9eae-84c23a2f93bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:16.964517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-13ee4528-bd61-47fe-9eae-84c23a2f93bd None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:16.966624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-13ee4528-bd61-47fe-9eae-84c23a2f93bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:17.152266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 116/236] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:17.352026 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13d250-f32b-4055-8691-4b3336efdaab None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:17.352489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13d250-f32b-4055-8691-4b3336efdaab None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:17.352629 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13d250-f32b-4055-8691-4b3336efdaab None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:17.353968 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 121/237] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:25:17.362672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70192f7-e747-402f-aa02-795c0480d6cd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:17.363081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70192f7-e747-402f-aa02-795c0480d6cd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:17.363304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70192f7-e747-402f-aa02-795c0480d6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:17.401063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e70192f7-e747-402f-aa02-795c0480d6cd None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:17.403208 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e70192f7-e747-402f-aa02-795c0480d6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:17.463817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13d250-f32b-4055-8691-4b3336efdaab None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:17.537901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 117/238] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:17.539264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92650e-3528-4737-aa0d-7e8161773c63 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:17.539544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92650e-3528-4737-aa0d-7e8161773c63 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:17.539839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92650e-3528-4737-aa0d-7e8161773c63 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:17.541371 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 118/239] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:17.546977 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23b3b7-1b4a-41c1-b1fe-ef02064ddfaf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:17.547265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23b3b7-1b4a-41c1-b1fe-ef02064ddfaf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:17.547527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23b3b7-1b4a-41c1-b1fe-ef02064ddfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:17.606956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bb23b3b7-1b4a-41c1-b1fe-ef02064ddfaf None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:17.612834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bb23b3b7-1b4a-41c1-b1fe-ef02064ddfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:17.744126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13d250-f32b-4055-8691-4b3336efdaab None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:17.758983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:17.784318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:17.787306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 119/240] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:17.827313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16e783-f284-42d6-8352-61478fb92a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:17.828369 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16e783-f284-42d6-8352-61478fb92a92 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:17.828882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16e783-f284-42d6-8352-61478fb92a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:17.829356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16e783-f284-42d6-8352-61478fb92a92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:17.835269 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16e783-f284-42d6-8352-61478fb92a92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:17.837166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16e783-f284-42d6-8352-61478fb92a92 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:17.839596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 122/241] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:25:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:17.849040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23b3b7-1b4a-41c1-b1fe-ef02064ddfaf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:18.046037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23b3b7-1b4a-41c1-b1fe-ef02064ddfaf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:18.104192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea9fd8-1152-40f1-a8d4-594a0a9b3362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:18.104829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea9fd8-1152-40f1-a8d4-594a0a9b3362 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:18.105105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea9fd8-1152-40f1-a8d4-594a0a9b3362 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:18.105374 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea9fd8-1152-40f1-a8d4-594a0a9b3362 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:18.106046 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea9fd8-1152-40f1-a8d4-594a0a9b3362 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:18.107657 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea9fd8-1152-40f1-a8d4-594a0a9b3362 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:18.193613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16e783-f284-42d6-8352-61478fb92a92 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:18.233015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 120/242] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:25:17 2022] POST /identity/v3/users => generated 293 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:25:18.375908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16e783-f284-42d6-8352-61478fb92a92 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:18.391302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea9fd8-1152-40f1-a8d4-594a0a9b3362 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:18.442447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7365323-4015-4834-abc5-0c7e050cbe99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:18.443537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7365323-4015-4834-abc5-0c7e050cbe99 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:18.444009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7365323-4015-4834-abc5-0c7e050cbe99 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:18.444479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7365323-4015-4834-abc5-0c7e050cbe99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:18.465705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7365323-4015-4834-abc5-0c7e050cbe99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d7365323-4015-4834-abc5-0c7e050cbe99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:18.477120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:18.478063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:18.478933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:18.478933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:18.478933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:18.478933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:18.478933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:18.478933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:18.478933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:18.478933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:18.478933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:18.478933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:18.478933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:18.478933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:18.480160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 123/243] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:18.487665 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7365323-4015-4834-abc5-0c7e050cbe99 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:18.522181 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:18.585455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:18.643247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea9fd8-1152-40f1-a8d4-594a0a9b3362 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:18.713147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2eb4e-99fa-4cba-bc6a-be841906f7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:18.714684 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2eb4e-99fa-4cba-bc6a-be841906f7cd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:18.715225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2eb4e-99fa-4cba-bc6a-be841906f7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:18.715717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2eb4e-99fa-4cba-bc6a-be841906f7cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:18.719380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7365323-4015-4834-abc5-0c7e050cbe99 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:18.737157 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2eb4e-99fa-4cba-bc6a-be841906f7cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-dcb2eb4e-99fa-4cba-bc6a-be841906f7cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:18.754938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:18.755972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:18.756771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:18.756771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:18.756771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:18.756771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:18.756771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:18.756771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:18.756771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:18.756771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:18.756771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:18.756771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:18.756771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:18.756771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:18.759351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 121/244] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:18.768768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2eb4e-99fa-4cba-bc6a-be841906f7cd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:18.798178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403057f-474b-48a5-95f4-cb93e4819eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:18.799842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403057f-474b-48a5-95f4-cb93e4819eaf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:18.800478 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403057f-474b-48a5-95f4-cb93e4819eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:18.801392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403057f-474b-48a5-95f4-cb93e4819eaf None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:18.802718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403057f-474b-48a5-95f4-cb93e4819eaf None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:18.803273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403057f-474b-48a5-95f4-cb93e4819eaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:18.805996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403057f-474b-48a5-95f4-cb93e4819eaf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:18.824311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 124/245] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:18.836173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403057f-474b-48a5-95f4-cb93e4819eaf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:19.081034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403057f-474b-48a5-95f4-cb93e4819eaf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:19.082899 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2eb4e-99fa-4cba-bc6a-be841906f7cd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:19.094848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:19.099344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:19.141983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8736c2a-3f75-4803-9365-670160f3f49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:19.143152 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736c2a-3f75-4803-9365-670160f3f49f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:19.143584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736c2a-3f75-4803-9365-670160f3f49f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:19.144059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736c2a-3f75-4803-9365-670160f3f49f None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:19.145906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8736c2a-3f75-4803-9365-670160f3f49f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:19.147922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8736c2a-3f75-4803-9365-670160f3f49f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:19.151791 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8736c2a-3f75-4803-9365-670160f3f49f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:19.172145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 122/246] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:19.174314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507c088-0c4c-4637-aec7-ce5362030959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:19.174817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d84e74-96bd-40b5-a73a-fcfe298c7f9d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:19.175305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d84e74-96bd-40b5-a73a-fcfe298c7f9d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:19.175838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d84e74-96bd-40b5-a73a-fcfe298c7f9d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:19.181296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507c088-0c4c-4637-aec7-ce5362030959 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:19.182064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507c088-0c4c-4637-aec7-ce5362030959 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:19.182596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507c088-0c4c-4637-aec7-ce5362030959 None admin] PATH_INFO: `/v3/users/d2491e3f0e3e483284c9d49659ecbaf4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:19.185983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507c088-0c4c-4637-aec7-ce5362030959 None admin] RBAC: Authorizing `identity:get_user(user_id=d2491e3f0e3e483284c9d49659ecbaf4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:19.187246 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 123/247] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:19 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:19.188839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d84e74-96bd-40b5-a73a-fcfe298c7f9d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:19.189011 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507c088-0c4c-4637-aec7-ce5362030959 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:19.193172 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 125/248] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:18 2022] GET /identity/v3/users/d2491e3f0e3e483284c9d49659ecbaf4 => generated 297 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:19.199071 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89b009-b331-4bb2-bf87-1aaf309dd4ba None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:19.200024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89b009-b331-4bb2-bf87-1aaf309dd4ba None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:19.203988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89b009-b331-4bb2-bf87-1aaf309dd4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:19.245192 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3a89b009-b331-4bb2-bf87-1aaf309dd4ba None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:19.247159 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a89b009-b331-4bb2-bf87-1aaf309dd4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:19.393173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 126/249] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:19.395597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89b009-b331-4bb2-bf87-1aaf309dd4ba None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:19.483113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d84e74-96bd-40b5-a73a-fcfe298c7f9d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:19.564450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89b009-b331-4bb2-bf87-1aaf309dd4ba None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:19.582962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb634815-ed39-4669-80c8-6b0367311f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:19.583630 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb634815-ed39-4669-80c8-6b0367311f30 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:19.583893 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb634815-ed39-4669-80c8-6b0367311f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:19.584168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb634815-ed39-4669-80c8-6b0367311f30 None admin] PATH_INFO: `/v3/users/83cd3ce4c3e449eaa1d7ffe84a33b96c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:19.598615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:19.614456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb634815-ed39-4669-80c8-6b0367311f30 None admin] RBAC: Authorizing `identity:get_user(user_id=83cd3ce4c3e449eaa1d7ffe84a33b96c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:19.617424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb634815-ed39-4669-80c8-6b0367311f30 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:19.625542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 124/250] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:19 2022] GET /identity/v3/users/83cd3ce4c3e449eaa1d7ffe84a33b96c => generated 297 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:19.631763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb634815-ed39-4669-80c8-6b0367311f30 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:19.644559 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70c6f6-5f06-4c25-ada0-3a00f7b7a10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:19.645459 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70c6f6-5f06-4c25-ada0-3a00f7b7a10c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:19.645921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70c6f6-5f06-4c25-ada0-3a00f7b7a10c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:19.646497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70c6f6-5f06-4c25-ada0-3a00f7b7a10c None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:19.648913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70c6f6-5f06-4c25-ada0-3a00f7b7a10c None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:19.652468 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70c6f6-5f06-4c25-ada0-3a00f7b7a10c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:19.662181 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 127/251] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:19 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:19.666206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70c6f6-5f06-4c25-ada0-3a00f7b7a10c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:19.723364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:19.873121 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70c6f6-5f06-4c25-ada0-3a00f7b7a10c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:19.947496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce7e22-77a2-48e9-8ade-374a899103ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:19.948515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce7e22-77a2-48e9-8ade-374a899103ee None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:19.948988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce7e22-77a2-48e9-8ade-374a899103ee None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:19.949448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce7e22-77a2-48e9-8ade-374a899103ee None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920/users/d2491e3f0e3e483284c9d49659ecbaf4/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:19.952891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce7e22-77a2-48e9-8ade-374a899103ee None admin] RBAC: Authorizing `identity:create_grant(project_id=ac74b750ef33436a83f5ee0474a71920, user_id=d2491e3f0e3e483284c9d49659ecbaf4, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:19.955316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce7e22-77a2-48e9-8ade-374a899103ee None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:19.961893 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb634815-ed39-4669-80c8-6b0367311f30 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:20.032541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467f83f-8d4c-4185-a573-62c7c7624c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:20.033775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f83f-8d4c-4185-a573-62c7c7624c4d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:20.034264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f83f-8d4c-4185-a573-62c7c7624c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:20.038712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f83f-8d4c-4185-a573-62c7c7624c4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:20.041508 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 128/252] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:25:19 2022] PUT /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920/users/d2491e3f0e3e483284c9d49659ecbaf4/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:25:20.042651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce7e22-77a2-48e9-8ade-374a899103ee None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:20.114011 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:20.250543 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467f83f-8d4c-4185-a573-62c7c7624c4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:20.253114 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467f83f-8d4c-4185-a573-62c7c7624c4d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:20.304698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce7e22-77a2-48e9-8ade-374a899103ee None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:20.360773 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:20.365353 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750c6f5-2dad-4aa7-a71d-642b6cf41cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:20.367420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750c6f5-2dad-4aa7-a71d-642b6cf41cab None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:20.367420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750c6f5-2dad-4aa7-a71d-642b6cf41cab None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:20.367420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750c6f5-2dad-4aa7-a71d-642b6cf41cab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:20.392118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750c6f5-2dad-4aa7-a71d-642b6cf41cab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e750c6f5-2dad-4aa7-a71d-642b6cf41cab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:20.397211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:20.399003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 129/253] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:25:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:20.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dab15a-32fb-4bf6-8259-2a93add8cf10 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:20.404087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dab15a-32fb-4bf6-8259-2a93add8cf10 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:20.404087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dab15a-32fb-4bf6-8259-2a93add8cf10 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:20.409015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 130/254] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:20.409924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dab15a-32fb-4bf6-8259-2a93add8cf10 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:20.540615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 125/255] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 910 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:20.542009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7b57c-38a0-401d-9aaf-f1152f704650 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:20.542330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7b57c-38a0-401d-9aaf-f1152f704650 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:20.542594 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7b57c-38a0-401d-9aaf-f1152f704650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:20.591989 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4ab7b57c-38a0-401d-9aaf-f1152f704650 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:20.594027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4ab7b57c-38a0-401d-9aaf-f1152f704650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:20.594439 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:20.612042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dab15a-32fb-4bf6-8259-2a93add8cf10 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:20.680564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8a509-d1f0-4d08-baa1-29016eb71657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:20.681239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8a509-d1f0-4d08-baa1-29016eb71657 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:20.681515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8a509-d1f0-4d08-baa1-29016eb71657 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:20.681782 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8a509-d1f0-4d08-baa1-29016eb71657 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:20.682619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8a509-d1f0-4d08-baa1-29016eb71657 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:20.682916 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8a509-d1f0-4d08-baa1-29016eb71657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:20.685319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8a509-d1f0-4d08-baa1-29016eb71657 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:20.704844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 131/256] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:20.705512 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8a509-d1f0-4d08-baa1-29016eb71657 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:20.742703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 126/257] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:20.743796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7b57c-38a0-401d-9aaf-f1152f704650 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:20.898380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:20.974170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8a509-d1f0-4d08-baa1-29016eb71657 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:21.035906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7b57c-38a0-401d-9aaf-f1152f704650 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:21.072051 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb98b7e-05dd-4aca-9494-a55f6c8f0a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:21.072512 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb98b7e-05dd-4aca-9494-a55f6c8f0a68 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:21.072773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb98b7e-05dd-4aca-9494-a55f6c8f0a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:21.073013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb98b7e-05dd-4aca-9494-a55f6c8f0a68 None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:21.079109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb98b7e-05dd-4aca-9494-a55f6c8f0a68 None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:21.080674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb98b7e-05dd-4aca-9494-a55f6c8f0a68 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:21.083095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 132/258] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:20 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:21.089466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb98b7e-05dd-4aca-9494-a55f6c8f0a68 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:21.143697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:21.155432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ed079-5106-4f69-8eb8-1a7eda719b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:21.156942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ed079-5106-4f69-8eb8-1a7eda719b11 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:21.157643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ed079-5106-4f69-8eb8-1a7eda719b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:21.161023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ed079-5106-4f69-8eb8-1a7eda719b11 None admin] PATH_INFO: `/v3/users/3691aa3013b84bc29d308fafe6c9b222` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:21.162528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ed079-5106-4f69-8eb8-1a7eda719b11 None admin] RBAC: Authorizing `identity:get_user(user_id=3691aa3013b84bc29d308fafe6c9b222)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:21.165596 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ed079-5106-4f69-8eb8-1a7eda719b11 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:21.169119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 127/259] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:20 2022] GET /identity/v3/users/3691aa3013b84bc29d308fafe6c9b222 => generated 283 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:21.176659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ed079-5106-4f69-8eb8-1a7eda719b11 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:21.314324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb98b7e-05dd-4aca-9494-a55f6c8f0a68 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:21.380325 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154afb7a-f562-4086-96db-2a2b678f10da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:21.380956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154afb7a-f562-4086-96db-2a2b678f10da None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:21.381232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154afb7a-f562-4086-96db-2a2b678f10da None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:21.381498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154afb7a-f562-4086-96db-2a2b678f10da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:21.385907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154afb7a-f562-4086-96db-2a2b678f10da None admin] RBAC: Adding query filter params (role.id=2684d96517574aa7bb00050a3c0e2d92, scope.project.id=ac74b750ef33436a83f5ee0474a71920, user.id=83cd3ce4c3e449eaa1d7ffe84a33b96c) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:21.385907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154afb7a-f562-4086-96db-2a2b678f10da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:21.385907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154afb7a-f562-4086-96db-2a2b678f10da None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:21.394897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 133/260] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:25:21 2022] GET /identity/v3/role_assignments?user.id=83cd3ce4c3e449eaa1d7ffe84a33b96c&scope.project.id=ac74b750ef33436a83f5ee0474a71920&role.id=2684d96517574aa7bb00050a3c0e2d92 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:21.395375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154afb7a-f562-4086-96db-2a2b678f10da None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:21.481796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ed079-5106-4f69-8eb8-1a7eda719b11 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:21.538556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:21.576160 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34c0d7-91a2-4eb2-9c81-2c201422eccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:21.576820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34c0d7-91a2-4eb2-9c81-2c201422eccb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:21.577095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34c0d7-91a2-4eb2-9c81-2c201422eccb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:21.577363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34c0d7-91a2-4eb2-9c81-2c201422eccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:21.610566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:21.647059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154afb7a-f562-4086-96db-2a2b678f10da None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:21.759052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee4b46-dbbc-425f-afa7-6d6460afe57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:21.759714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee4b46-dbbc-425f-afa7-6d6460afe57d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:21.759992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee4b46-dbbc-425f-afa7-6d6460afe57d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:21.760263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee4b46-dbbc-425f-afa7-6d6460afe57d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:21.781740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34c0d7-91a2-4eb2-9c81-2c201422eccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:21.784364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34c0d7-91a2-4eb2-9c81-2c201422eccb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:21.795360 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee4b46-dbbc-425f-afa7-6d6460afe57d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5eee4b46-dbbc-425f-afa7-6d6460afe57d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:21.801555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:21.802558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:21.803361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:21.803361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:21.803361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:21.803361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:21.803361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:21.803361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:21.803361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:21.803361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:21.803361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:21.803361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:21.803361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:21.803361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:21.807416 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 134/261] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:25:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:21.817697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee4b46-dbbc-425f-afa7-6d6460afe57d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:21.999788 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee4b46-dbbc-425f-afa7-6d6460afe57d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:22.054607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 128/262] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 879 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 08:25:22.056029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8838e6-2220-49f5-b7c8-392862fd65f0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:22.056309 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8838e6-2220-49f5-b7c8-392862fd65f0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:22.056578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8838e6-2220-49f5-b7c8-392862fd65f0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:22.059795 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 129/263] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:22.067439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8838e6-2220-49f5-b7c8-392862fd65f0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:22.073153 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093d566-e2a9-4560-9c7e-74549962d6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:22.073642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093d566-e2a9-4560-9c7e-74549962d6db None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:22.073915 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093d566-e2a9-4560-9c7e-74549962d6db None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:22.074203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093d566-e2a9-4560-9c7e-74549962d6db None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:22.074897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093d566-e2a9-4560-9c7e-74549962d6db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:22.075202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093d566-e2a9-4560-9c7e-74549962d6db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:22.076634 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093d566-e2a9-4560-9c7e-74549962d6db None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:22.104082 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 135/264] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:25:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:22.105406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0c5d8-a007-4b29-8959-f4480b6689cc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:22.105693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0c5d8-a007-4b29-8959-f4480b6689cc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:22.105966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0c5d8-a007-4b29-8959-f4480b6689cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:22.125746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:22.136873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-99b0c5d8-a007-4b29-8959-f4480b6689cc None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:22.138108 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-99b0c5d8-a007-4b29-8959-f4480b6689cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:22.144276 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:22.244080 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8838e6-2220-49f5-b7c8-392862fd65f0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:22.287965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481d69d-1deb-4e1b-85cd-ae46fcd9cc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:22.288597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481d69d-1deb-4e1b-85cd-ae46fcd9cc5e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:22.288924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481d69d-1deb-4e1b-85cd-ae46fcd9cc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:22.289198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481d69d-1deb-4e1b-85cd-ae46fcd9cc5e None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:22.291637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481d69d-1deb-4e1b-85cd-ae46fcd9cc5e None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:22.293124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481d69d-1deb-4e1b-85cd-ae46fcd9cc5e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:22.296874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 130/265] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:22 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:22.297751 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481d69d-1deb-4e1b-85cd-ae46fcd9cc5e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:22.325485 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 136/266] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:22.326398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0c5d8-a007-4b29-8959-f4480b6689cc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:22.430200 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:22.443468 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481d69d-1deb-4e1b-85cd-ae46fcd9cc5e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:22.448526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0c5d8-a007-4b29-8959-f4480b6689cc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:22.489941 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8b05c-a252-4814-abe3-6128d68ade15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:22.490657 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8b05c-a252-4814-abe3-6128d68ade15 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:22.490937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8b05c-a252-4814-abe3-6128d68ade15 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:22.491125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6961224-328f-4c03-82eb-f0ee881b6489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:22.491240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8b05c-a252-4814-abe3-6128d68ade15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:22.491763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6961224-328f-4c03-82eb-f0ee881b6489 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:22.491933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8b05c-a252-4814-abe3-6128d68ade15 None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=ac74b750ef33436a83f5ee0474a71920, user.id=3691aa3013b84bc29d308fafe6c9b222) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:22.492030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6961224-328f-4c03-82eb-f0ee881b6489 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:22.492241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8b05c-a252-4814-abe3-6128d68ade15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:22.492338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6961224-328f-4c03-82eb-f0ee881b6489 None admin] PATH_INFO: `/v3/users/3691aa3013b84bc29d308fafe6c9b222` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:22.493507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6961224-328f-4c03-82eb-f0ee881b6489 None admin] RBAC: Authorizing `identity:get_user(user_id=3691aa3013b84bc29d308fafe6c9b222)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:22.493687 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8b05c-a252-4814-abe3-6128d68ade15 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:22.494997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6961224-328f-4c03-82eb-f0ee881b6489 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:22.497934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 131/267] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:22 2022] GET /identity/v3/users/3691aa3013b84bc29d308fafe6c9b222 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:22.503443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75ae1b-c975-4a95-957c-924c4111a7ff None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:22.503747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75ae1b-c975-4a95-957c-924c4111a7ff None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:22.503993 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75ae1b-c975-4a95-957c-924c4111a7ff None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:22.505637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 132/268] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:22.507109 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75ae1b-c975-4a95-957c-924c4111a7ff None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:22.508600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 137/269] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:25:22 2022] GET /identity/v3/role_assignments?user.id=3691aa3013b84bc29d308fafe6c9b222&scope.project.id=ac74b750ef33436a83f5ee0474a71920&role.id=020bc02dca71485bab2312442adaa179 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:22.525980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f8c78-cf13-4a5f-bb3e-10cc97766e11 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:22.529887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f8c78-cf13-4a5f-bb3e-10cc97766e11 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:22.529887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f8c78-cf13-4a5f-bb3e-10cc97766e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:22.534100 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:22.557204 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dc8f8c78-cf13-4a5f-bb3e-10cc97766e11 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:22.558474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dc8f8c78-cf13-4a5f-bb3e-10cc97766e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:22.731613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 138/270] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:22.791469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75ae1b-c975-4a95-957c-924c4111a7ff None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:22.872032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81413c41-2336-47d6-9b9f-a47c8d61e135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:22.872582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81413c41-2336-47d6-9b9f-a47c8d61e135 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:22.872853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81413c41-2336-47d6-9b9f-a47c8d61e135 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:22.873118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81413c41-2336-47d6-9b9f-a47c8d61e135 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:22.916183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:22.943899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f8c78-cf13-4a5f-bb3e-10cc97766e11 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:23.013278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81413c41-2336-47d6-9b9f-a47c8d61e135 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:23.013278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81413c41-2336-47d6-9b9f-a47c8d61e135 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:23.132324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f8c78-cf13-4a5f-bb3e-10cc97766e11 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:23.146588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 133/271] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:23.153187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81413c41-2336-47d6-9b9f-a47c8d61e135 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:23.179720 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:23.197085 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8062f-4d71-4811-90c0-375f68a0c153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:23.197740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8062f-4d71-4811-90c0-375f68a0c153 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:23.198017 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8062f-4d71-4811-90c0-375f68a0c153 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:23.198313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8062f-4d71-4811-90c0-375f68a0c153 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:23.202986 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8062f-4d71-4811-90c0-375f68a0c153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:23.204511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8062f-4d71-4811-90c0-375f68a0c153 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:23.207190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 139/272] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:25:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:23.212623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8062f-4d71-4811-90c0-375f68a0c153 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:23.229460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:23.314496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81413c41-2336-47d6-9b9f-a47c8d61e135 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:23.380841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fe998-0630-4c2a-bc2e-701e8bfafe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:23.382511 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fe998-0630-4c2a-bc2e-701e8bfafe24 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:23.383259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fe998-0630-4c2a-bc2e-701e8bfafe24 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:23.383972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fe998-0630-4c2a-bc2e-701e8bfafe24 None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:23.386542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fe998-0630-4c2a-bc2e-701e8bfafe24 None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:23.392719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fe998-0630-4c2a-bc2e-701e8bfafe24 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:23.396134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 134/273] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:23 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:23.397483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fe998-0630-4c2a-bc2e-701e8bfafe24 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:23.463278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8062f-4d71-4811-90c0-375f68a0c153 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:23.524153 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf72c3-2cd7-45e0-9005-668b176dc852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:23.525448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf72c3-2cd7-45e0-9005-668b176dc852 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:23.525448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf72c3-2cd7-45e0-9005-668b176dc852 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:23.525448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf72c3-2cd7-45e0-9005-668b176dc852 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:23.526075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf72c3-2cd7-45e0-9005-668b176dc852 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:23.528353 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf72c3-2cd7-45e0-9005-668b176dc852 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:23.578801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fe998-0630-4c2a-bc2e-701e8bfafe24 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:23.611432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:23.621196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65013269-0895-4461-853a-0e851edea874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:23.621971 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65013269-0895-4461-853a-0e851edea874 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:23.622375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65013269-0895-4461-853a-0e851edea874 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:23.622744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65013269-0895-4461-853a-0e851edea874 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:23.634990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65013269-0895-4461-853a-0e851edea874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-65013269-0895-4461-853a-0e851edea874 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:23.640145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:23.642321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 135/274] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:23.644984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65013269-0895-4461-853a-0e851edea874 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:23.837103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65013269-0895-4461-853a-0e851edea874 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:23.879132 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 140/275] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:25:23 2022] POST /identity/v3/users => generated 293 bytes in 667 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:25:23.881120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9755b-9fd6-423c-9540-9221fa183950 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:23.881688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9755b-9fd6-423c-9540-9221fa183950 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:23.882592 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9755b-9fd6-423c-9540-9221fa183950 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:23.884695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 141/276] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:23.892208 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9755b-9fd6-423c-9540-9221fa183950 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:23.908068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d611879-8580-44dd-b4dd-02adb3fc54c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:23.909373 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d611879-8580-44dd-b4dd-02adb3fc54c6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:23.910558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d611879-8580-44dd-b4dd-02adb3fc54c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:23.911106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d611879-8580-44dd-b4dd-02adb3fc54c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:23.912659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d611879-8580-44dd-b4dd-02adb3fc54c6 None admin] RBAC: Adding query filter params (role.id=65f33fe3045c433b9a3e045e7572aed6, scope.project.id=ac74b750ef33436a83f5ee0474a71920, user.id=3691aa3013b84bc29d308fafe6c9b222) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:23.913330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d611879-8580-44dd-b4dd-02adb3fc54c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:23.916716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d611879-8580-44dd-b4dd-02adb3fc54c6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:23.929878 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 136/277] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:25:23 2022] GET /identity/v3/role_assignments?user.id=3691aa3013b84bc29d308fafe6c9b222&scope.project.id=ac74b750ef33436a83f5ee0474a71920&role.id=65f33fe3045c433b9a3e045e7572aed6 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:23.932366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd95ae0-ea98-49ab-b1d2-ad657b103b36 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:23.933160 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd95ae0-ea98-49ab-b1d2-ad657b103b36 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:23.933719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd95ae0-ea98-49ab-b1d2-ad657b103b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:23.969353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cfd95ae0-ea98-49ab-b1d2-ad657b103b36 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:23.971837 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cfd95ae0-ea98-49ab-b1d2-ad657b103b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:24.011654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:24.108263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9755b-9fd6-423c-9540-9221fa183950 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:24.128003 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:24.168901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 137/278] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:24.180764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce580713-9957-4999-9668-5bc9df6677c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:24.181767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce580713-9957-4999-9668-5bc9df6677c7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:24.182269 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce580713-9957-4999-9668-5bc9df6677c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:24.182732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce580713-9957-4999-9668-5bc9df6677c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:24.183894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce580713-9957-4999-9668-5bc9df6677c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:24.184407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce580713-9957-4999-9668-5bc9df6677c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:24.190283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce580713-9957-4999-9668-5bc9df6677c7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:24.211983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 142/279] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:24.221175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd95ae0-ea98-49ab-b1d2-ad657b103b36 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:24.431399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd95ae0-ea98-49ab-b1d2-ad657b103b36 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:24.500722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9ca25-102a-4986-8fe6-d6dc57dc40db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:24.501878 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9ca25-102a-4986-8fe6-d6dc57dc40db None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:24.502579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9ca25-102a-4986-8fe6-d6dc57dc40db None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:24.503015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9ca25-102a-4986-8fe6-d6dc57dc40db None admin] PATH_INFO: `/v3/users/d5e45f5cc6354082af5b120c87b64d61` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:24.532110 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9ca25-102a-4986-8fe6-d6dc57dc40db None admin] RBAC: Authorizing `identity:get_user(user_id=d5e45f5cc6354082af5b120c87b64d61)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:24.535066 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9ca25-102a-4986-8fe6-d6dc57dc40db None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:24.539246 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 138/280] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:24 2022] GET /identity/v3/users/d5e45f5cc6354082af5b120c87b64d61 => generated 293 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:24.547942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce580713-9957-4999-9668-5bc9df6677c7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:24.743022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9ca25-102a-4986-8fe6-d6dc57dc40db None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:24.775350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce580713-9957-4999-9668-5bc9df6677c7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:24.819978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:24.841008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:24.844697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd15a7-8ae1-4044-9187-cd88d93da592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:24.847242 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd15a7-8ae1-4044-9187-cd88d93da592 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:24.847242 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd15a7-8ae1-4044-9187-cd88d93da592 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:24.847242 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd15a7-8ae1-4044-9187-cd88d93da592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:24.847242 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd15a7-8ae1-4044-9187-cd88d93da592 None admin] RBAC: Adding query filter params (role.id=dba75ce7d709483fbbb0027d29e99725, user.id=d5e45f5cc6354082af5b120c87b64d61, scope.system=all) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:24.847242 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd15a7-8ae1-4044-9187-cd88d93da592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:24.848444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd15a7-8ae1-4044-9187-cd88d93da592 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:24.855217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 143/281] 10.222.0.44 () {56 vars in 1308 bytes} [Sun Sep 11 08:25:24 2022] GET /identity/v3/role_assignments?user.id=d5e45f5cc6354082af5b120c87b64d61&scope.system=all&role.id=dba75ce7d709483fbbb0027d29e99725 => generated 226 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:24.982448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9ca25-102a-4986-8fe6-d6dc57dc40db None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:25.052045 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0ddd8-7881-4cee-81c5-74c7620adfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:25.053430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0ddd8-7881-4cee-81c5-74c7620adfcb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:25.054090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0ddd8-7881-4cee-81c5-74c7620adfcb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:25.054799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0ddd8-7881-4cee-81c5-74c7620adfcb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:25.076906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0ddd8-7881-4cee-81c5-74c7620adfcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-fbd0ddd8-7881-4cee-81c5-74c7620adfcb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:25.087957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:25.090306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:25.092266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:25.092266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:25.092266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:25.092266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:25.092266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:25.092266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:25.092266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:25.092266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:25.092266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:25.092266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:25.092266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:25.092266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:25.096490 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 139/282] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:25.107377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd15a7-8ae1-4044-9187-cd88d93da592 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:25.253734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd15a7-8ae1-4044-9187-cd88d93da592 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:25.298175 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc562b-ab3c-4e53-9d27-d429698dd6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:25.298809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc562b-ab3c-4e53-9d27-d429698dd6df None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:25.299087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc562b-ab3c-4e53-9d27-d429698dd6df None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:25.299356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc562b-ab3c-4e53-9d27-d429698dd6df None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:25.300033 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc562b-ab3c-4e53-9d27-d429698dd6df None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:25.300903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc562b-ab3c-4e53-9d27-d429698dd6df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:25.302423 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc562b-ab3c-4e53-9d27-d429698dd6df None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:25.317157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 144/283] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:25.323817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0ddd8-7881-4cee-81c5-74c7620adfcb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:25.595425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:25.599453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0ddd8-7881-4cee-81c5-74c7620adfcb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:25.681959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d1f03-5fcd-404b-bad2-14dc07afe88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:25.683083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d1f03-5fcd-404b-bad2-14dc07afe88b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:25.683514 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d1f03-5fcd-404b-bad2-14dc07afe88b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:25.684014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d1f03-5fcd-404b-bad2-14dc07afe88b None admin] PATH_INFO: `/v3/users/d2491e3f0e3e483284c9d49659ecbaf4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:25.690359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d1f03-5fcd-404b-bad2-14dc07afe88b None admin] RBAC: Authorizing `identity:get_user(user_id=d2491e3f0e3e483284c9d49659ecbaf4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:25.693219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d1f03-5fcd-404b-bad2-14dc07afe88b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:25.696716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 140/284] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:25 2022] GET /identity/v3/users/d2491e3f0e3e483284c9d49659ecbaf4 => generated 297 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:25.705029 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc562b-ab3c-4e53-9d27-d429698dd6df None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:25.733212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784f448-e05b-4985-b5fb-3af5e9ba645a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:25.733714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784f448-e05b-4985-b5fb-3af5e9ba645a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:25.734092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784f448-e05b-4985-b5fb-3af5e9ba645a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:25.738649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 141/285] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:25:25.752638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd986054-4c72-4fc7-a20c-f332c865fa7f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:25.753151 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd986054-4c72-4fc7-a20c-f332c865fa7f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:25.753696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd986054-4c72-4fc7-a20c-f332c865fa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:25.815647 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-dd986054-4c72-4fc7-a20c-f332c865fa7f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:25.819178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dd986054-4c72-4fc7-a20c-f332c865fa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:25.838714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:25.938830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc562b-ab3c-4e53-9d27-d429698dd6df None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:25.979833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 142/286] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:26.004321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fb6b9-c382-4e13-9b48-19791fad02c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:26.005214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fb6b9-c382-4e13-9b48-19791fad02c5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:26.005664 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fb6b9-c382-4e13-9b48-19791fad02c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:26.006129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fb6b9-c382-4e13-9b48-19791fad02c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:26.126787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fb6b9-c382-4e13-9b48-19791fad02c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:26.130581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fb6b9-c382-4e13-9b48-19791fad02c5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:26.237313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:26.326494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd986054-4c72-4fc7-a20c-f332c865fa7f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:26.375977 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 145/287] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:26.386733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fb6b9-c382-4e13-9b48-19791fad02c5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:26.640391 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:26.647894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd986054-4c72-4fc7-a20c-f332c865fa7f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:26.673241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fb6b9-c382-4e13-9b48-19791fad02c5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:26.726547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfcf14-62cd-42fc-9f34-ce7cde78ff10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:26.728064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfcf14-62cd-42fc-9f34-ce7cde78ff10 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:26.728806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfcf14-62cd-42fc-9f34-ce7cde78ff10 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:26.729618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfcf14-62cd-42fc-9f34-ce7cde78ff10 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:26.760906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bfcf14-62cd-42fc-9f34-ce7cde78ff10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f9bfcf14-62cd-42fc-9f34-ce7cde78ff10 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:26.773067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:26.774969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:26.776528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:26.776528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:26.776528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:26.776528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:26.776528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:26.776528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:26.776528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:26.776528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:26.776528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:26.776528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:26.776528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:26.776528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:26.781437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efadc7d-5242-4c0b-b101-4f920d7a2e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:26.783512 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efadc7d-5242-4c0b-b101-4f920d7a2e41 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:26.784289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efadc7d-5242-4c0b-b101-4f920d7a2e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:26.785643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efadc7d-5242-4c0b-b101-4f920d7a2e41 None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:26.788095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efadc7d-5242-4c0b-b101-4f920d7a2e41 None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:26.791378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efadc7d-5242-4c0b-b101-4f920d7a2e41 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:26.797120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 146/288] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:26 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:26.797813 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 143/289] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 472 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:25:26.803720 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfcf14-62cd-42fc-9f34-ce7cde78ff10 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:26.812589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efadc7d-5242-4c0b-b101-4f920d7a2e41 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:26.842918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:27.054083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfcf14-62cd-42fc-9f34-ce7cde78ff10 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:27.086752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efadc7d-5242-4c0b-b101-4f920d7a2e41 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:27.150877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875abb60-29ab-4e64-ba06-a9aa85a0c3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:27.151720 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875abb60-29ab-4e64-ba06-a9aa85a0c3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:27.152042 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875abb60-29ab-4e64-ba06-a9aa85a0c3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:27.152328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875abb60-29ab-4e64-ba06-a9aa85a0c3c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:27.153319 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875abb60-29ab-4e64-ba06-a9aa85a0c3c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:27.153610 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875abb60-29ab-4e64-ba06-a9aa85a0c3c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:27.155437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875abb60-29ab-4e64-ba06-a9aa85a0c3c2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:27.163761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c366e-563d-4d46-a0cb-452da824d602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:27.164761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c366e-563d-4d46-a0cb-452da824d602 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:27.165058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c366e-563d-4d46-a0cb-452da824d602 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:27.165365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c366e-563d-4d46-a0cb-452da824d602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:27.166173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c366e-563d-4d46-a0cb-452da824d602 None admin] RBAC: Adding query filter params (role.id=dba75ce7d709483fbbb0027d29e99725, scope.project.id=ac74b750ef33436a83f5ee0474a71920, user.id=d2491e3f0e3e483284c9d49659ecbaf4) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:27.166502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c366e-563d-4d46-a0cb-452da824d602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:27.168167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c366e-563d-4d46-a0cb-452da824d602 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:27.180933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 147/290] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:25:26 2022] GET /identity/v3/role_assignments?user.id=d2491e3f0e3e483284c9d49659ecbaf4&scope.project.id=ac74b750ef33436a83f5ee0474a71920&role.id=dba75ce7d709483fbbb0027d29e99725 => generated 608 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:27.181331 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 144/291] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:27.187996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c366e-563d-4d46-a0cb-452da824d602 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:27.352879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:27.399951 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c366e-563d-4d46-a0cb-452da824d602 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:27.463602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8ca2b-2094-4512-95c2-c00c3e4e3f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:27.464718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8ca2b-2094-4512-95c2-c00c3e4e3f08 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:27.464995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8ca2b-2094-4512-95c2-c00c3e4e3f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:27.465373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8ca2b-2094-4512-95c2-c00c3e4e3f08 None admin] PATH_INFO: `/v3/users/83cd3ce4c3e449eaa1d7ffe84a33b96c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:27.466868 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8ca2b-2094-4512-95c2-c00c3e4e3f08 None admin] RBAC: Authorizing `identity:get_user(user_id=83cd3ce4c3e449eaa1d7ffe84a33b96c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:27.468841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8ca2b-2094-4512-95c2-c00c3e4e3f08 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:27.471057 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 148/292] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:27 2022] GET /identity/v3/users/83cd3ce4c3e449eaa1d7ffe84a33b96c => generated 297 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:27.477698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875abb60-29ab-4e64-ba06-a9aa85a0c3c2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:27.514512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:27.666760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875abb60-29ab-4e64-ba06-a9aa85a0c3c2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:27.719760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bee111-e15d-4320-880a-628158015bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:27.720408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bee111-e15d-4320-880a-628158015bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:27.720679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bee111-e15d-4320-880a-628158015bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:27.720945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bee111-e15d-4320-880a-628158015bc8 None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:27.722166 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bee111-e15d-4320-880a-628158015bc8 None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:27.723647 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bee111-e15d-4320-880a-628158015bc8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:27.725535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 145/293] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:27 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:27.731334 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8ca2b-2094-4512-95c2-c00c3e4e3f08 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:27.763474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f47b19-09b0-4ad1-b32e-d0b13a159fd5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:27.763938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f47b19-09b0-4ad1-b32e-d0b13a159fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:27.764369 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f47b19-09b0-4ad1-b32e-d0b13a159fd5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:27.768959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 146/294] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:27.776407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fda54c-16bf-4cfc-84fc-2c329e2593f9 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:27.776861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fda54c-16bf-4cfc-84fc-2c329e2593f9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:27.777325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fda54c-16bf-4cfc-84fc-2c329e2593f9 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:27.780251 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 147/295] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:27.782415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d8ee3-90b5-445a-923a-3912f1d1d75e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:27.782886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d8ee3-90b5-445a-923a-3912f1d1d75e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:27.783366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d8ee3-90b5-445a-923a-3912f1d1d75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:27.814955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-031d8ee3-90b5-445a-923a-3912f1d1d75e None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:27.817654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-031d8ee3-90b5-445a-923a-3912f1d1d75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:27.866166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8ca2b-2094-4512-95c2-c00c3e4e3f08 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:27.920466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87414bfc-1e2b-4333-a14f-b74a34c5b888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:27.921546 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87414bfc-1e2b-4333-a14f-b74a34c5b888 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:27.921916 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87414bfc-1e2b-4333-a14f-b74a34c5b888 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:27.922271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87414bfc-1e2b-4333-a14f-b74a34c5b888 None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920/users/83cd3ce4c3e449eaa1d7ffe84a33b96c/roles/2684d96517574aa7bb00050a3c0e2d92` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:27.925745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87414bfc-1e2b-4333-a14f-b74a34c5b888 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac74b750ef33436a83f5ee0474a71920, user_id=83cd3ce4c3e449eaa1d7ffe84a33b96c, role_id=2684d96517574aa7bb00050a3c0e2d92)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:27.927822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87414bfc-1e2b-4333-a14f-b74a34c5b888 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:27.952289 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 148/296] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:27.955133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449a46a-2db2-45a3-862f-b4f8d8d3ca9c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:27.955632 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449a46a-2db2-45a3-862f-b4f8d8d3ca9c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:27.956104 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449a46a-2db2-45a3-862f-b4f8d8d3ca9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:27.990095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1449a46a-2db2-45a3-862f-b4f8d8d3ca9c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:27.992379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1449a46a-2db2-45a3-862f-b4f8d8d3ca9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:28.021399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:28.036515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 149/297] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:25:27 2022] PUT /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920/users/83cd3ce4c3e449eaa1d7ffe84a33b96c/roles/2684d96517574aa7bb00050a3c0e2d92 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:25:28.084058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510802-70c5-445a-8b9c-a86aa46d4fc2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:28.084345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510802-70c5-445a-8b9c-a86aa46d4fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:28.084760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510802-70c5-445a-8b9c-a86aa46d4fc2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:28.085985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 150/298] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:25:28.092269 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74c998-889c-42a0-bed4-31f96577fa39 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:28.092943 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74c998-889c-42a0-bed4-31f96577fa39 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:28.092943 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74c998-889c-42a0-bed4-31f96577fa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:28.102913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:28.127829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3a74c998-889c-42a0-bed4-31f96577fa39 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:28.132596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a74c998-889c-42a0-bed4-31f96577fa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:28.137219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 149/299] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:28.244512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449a46a-2db2-45a3-862f-b4f8d8d3ca9c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:28.245581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 151/300] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:28.378895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449a46a-2db2-45a3-862f-b4f8d8d3ca9c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:28.411544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74c998-889c-42a0-bed4-31f96577fa39 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:28.429420 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb999f2-296b-4c63-9a42-abed3ef6075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:28.430601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb999f2-296b-4c63-9a42-abed3ef6075e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:28.431031 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb999f2-296b-4c63-9a42-abed3ef6075e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:28.432122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb999f2-296b-4c63-9a42-abed3ef6075e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:28.447368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb999f2-296b-4c63-9a42-abed3ef6075e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3bb999f2-296b-4c63-9a42-abed3ef6075e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:28.453401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:28.454658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:28.455671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:28.455671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:28.455671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:28.455671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:28.455671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:28.455671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:28.455671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:28.455671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:28.455671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:28.455671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:28.455671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:28.455671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:28.463581 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 150/301] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:28.465780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb999f2-296b-4c63-9a42-abed3ef6075e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:28.486634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:28.600904 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb999f2-296b-4c63-9a42-abed3ef6075e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:28.621439 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74c998-889c-42a0-bed4-31f96577fa39 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:28.644920 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c6865-f434-4a05-97f5-b87f958c2c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:28.645920 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c6865-f434-4a05-97f5-b87f958c2c44 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:28.646545 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c6865-f434-4a05-97f5-b87f958c2c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:28.646989 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c6865-f434-4a05-97f5-b87f958c2c44 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:28.647828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c6865-f434-4a05-97f5-b87f958c2c44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:28.648281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c6865-f434-4a05-97f5-b87f958c2c44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:28.649953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c6865-f434-4a05-97f5-b87f958c2c44 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:28.666401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 151/302] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:28.668113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c6865-f434-4a05-97f5-b87f958c2c44 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:28.694982 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20751a4-bfa8-4a3a-9e93-93e460c1e23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:28.696079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20751a4-bfa8-4a3a-9e93-93e460c1e23a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:28.696571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20751a4-bfa8-4a3a-9e93-93e460c1e23a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:28.697054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20751a4-bfa8-4a3a-9e93-93e460c1e23a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:28.699293 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20751a4-bfa8-4a3a-9e93-93e460c1e23a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:28.701644 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20751a4-bfa8-4a3a-9e93-93e460c1e23a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:28.705986 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 152/303] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:25:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:28.709472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20751a4-bfa8-4a3a-9e93-93e460c1e23a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:28.714840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:28.788281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:28.848621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c6865-f434-4a05-97f5-b87f958c2c44 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:28.861229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20751a4-bfa8-4a3a-9e93-93e460c1e23a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:28.913266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac6438-6127-45bc-baf8-9094a177c1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:28.915008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6438-6127-45bc-baf8-9094a177c1ab None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:28.915286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6438-6127-45bc-baf8-9094a177c1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:28.915654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6438-6127-45bc-baf8-9094a177c1ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:28.916428 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd5484-bd4f-44fc-ad56-0990ba8ca49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:28.917482 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd5484-bd4f-44fc-ad56-0990ba8ca49c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:28.918263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd5484-bd4f-44fc-ad56-0990ba8ca49c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:28.919198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd5484-bd4f-44fc-ad56-0990ba8ca49c None admin] PATH_INFO: `/v3/users/7e8931f249d049e38e860f7ab6bbffb6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:28.933179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac6438-6127-45bc-baf8-9094a177c1ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:28.935965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd5484-bd4f-44fc-ad56-0990ba8ca49c None admin] RBAC: Authorizing `identity:get_user(user_id=7e8931f249d049e38e860f7ab6bbffb6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:28.937430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd5484-bd4f-44fc-ad56-0990ba8ca49c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:28.939430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 152/304] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:28 2022] GET /identity/v3/users/7e8931f249d049e38e860f7ab6bbffb6 => generated 293 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:28.939940 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd5484-bd4f-44fc-ad56-0990ba8ca49c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b5ac6438-6127-45bc-baf8-9094a177c1ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:28.944884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:28.945847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:28.946677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:28.946677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:28.946677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:28.946677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:28.946677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:28.946677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:28.946677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:28.946677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:28.946677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:28.946677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:28.946677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:28.946677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:28.947329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 153/305] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:28.948206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac6438-6127-45bc-baf8-9094a177c1ab None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:29.069576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd5484-bd4f-44fc-ad56-0990ba8ca49c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:29.071981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:29.112115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:29.125629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b771a6-9610-4ab3-b0f1-257a44929566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:29.126654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b771a6-9610-4ab3-b0f1-257a44929566 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:29.127052 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b771a6-9610-4ab3-b0f1-257a44929566 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:29.127448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b771a6-9610-4ab3-b0f1-257a44929566 None admin] PATH_INFO: `/v3/groups` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:29.129070 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b771a6-9610-4ab3-b0f1-257a44929566 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:29.131578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b771a6-9610-4ab3-b0f1-257a44929566 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:29.144702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac6438-6127-45bc-baf8-9094a177c1ab None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:29.184901 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694715f-450c-44a5-b52d-e612739ab81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:29.185639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694715f-450c-44a5-b52d-e612739ab81c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:29.185971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694715f-450c-44a5-b52d-e612739ab81c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:29.186355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694715f-450c-44a5-b52d-e612739ab81c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:29.187228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694715f-450c-44a5-b52d-e612739ab81c None admin] RBAC: Adding query filter params (role.id=2684d96517574aa7bb00050a3c0e2d92, user.id=7e8931f249d049e38e860f7ab6bbffb6, scope.system=all) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:29.187590 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694715f-450c-44a5-b52d-e612739ab81c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:29.189095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694715f-450c-44a5-b52d-e612739ab81c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:29.195769 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 154/306] 10.222.0.44 () {56 vars in 1308 bytes} [Sun Sep 11 08:25:28 2022] GET /identity/v3/role_assignments?user.id=7e8931f249d049e38e860f7ab6bbffb6&scope.system=all&role.id=2684d96517574aa7bb00050a3c0e2d92 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:29.196959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694715f-450c-44a5-b52d-e612739ab81c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:29.246915 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 153/307] 10.222.0.44 () {60 vars in 1134 bytes} [Sun Sep 11 08:25:28 2022] POST /identity/v3/groups => generated 229 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:25:29.391910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694715f-450c-44a5-b52d-e612739ab81c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:29.438664 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:29.454322 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d7df8-e07d-4b3d-99c8-b38b8ca95a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:29.455317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d7df8-e07d-4b3d-99c8-b38b8ca95a7e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:29.455785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d7df8-e07d-4b3d-99c8-b38b8ca95a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:29.456244 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d7df8-e07d-4b3d-99c8-b38b8ca95a7e None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:29.457457 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d7df8-e07d-4b3d-99c8-b38b8ca95a7e None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:29.458041 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d7df8-e07d-4b3d-99c8-b38b8ca95a7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:29.460686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d7df8-e07d-4b3d-99c8-b38b8ca95a7e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:29.479210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 155/308] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:29.488600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b771a6-9610-4ab3-b0f1-257a44929566 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:29.678628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b771a6-9610-4ab3-b0f1-257a44929566 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:29.745771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbf1e1-220c-42c1-8a6a-88d279a5e300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:29.747278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbf1e1-220c-42c1-8a6a-88d279a5e300 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:29.748035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbf1e1-220c-42c1-8a6a-88d279a5e300 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:29.748722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbf1e1-220c-42c1-8a6a-88d279a5e300 None admin] PATH_INFO: `/v3/users/d5e45f5cc6354082af5b120c87b64d61` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:29.751087 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cbf1e1-220c-42c1-8a6a-88d279a5e300 None admin] RBAC: Authorizing `identity:get_user(user_id=d5e45f5cc6354082af5b120c87b64d61)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:29.754205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cbf1e1-220c-42c1-8a6a-88d279a5e300 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:29.758126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 154/309] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:29 2022] GET /identity/v3/users/d5e45f5cc6354082af5b120c87b64d61 => generated 293 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:29.766206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d7df8-e07d-4b3d-99c8-b38b8ca95a7e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:29.959206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d7df8-e07d-4b3d-99c8-b38b8ca95a7e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:30.033386 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15430bb-321c-4db6-b86f-59a970ff1e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:30.035059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15430bb-321c-4db6-b86f-59a970ff1e47 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:30.035541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15430bb-321c-4db6-b86f-59a970ff1e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:30.036015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15430bb-321c-4db6-b86f-59a970ff1e47 None admin] PATH_INFO: `/v3/system/users/d5e45f5cc6354082af5b120c87b64d61/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:30.038548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15430bb-321c-4db6-b86f-59a970ff1e47 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d5e45f5cc6354082af5b120c87b64d61, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:30.041126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15430bb-321c-4db6-b86f-59a970ff1e47 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:30.164334 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 156/310] 10.222.0.44 () {58 vars in 1334 bytes} [Sun Sep 11 08:25:29 2022] PUT /identity/v3/system/users/d5e45f5cc6354082af5b120c87b64d61/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:25:30.897180 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e7e41-3005-4a7a-bade-4aecda43e463 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:30.897844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e7e41-3005-4a7a-bade-4aecda43e463 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:30.897844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e7e41-3005-4a7a-bade-4aecda43e463 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:30.899436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 155/311] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:30.904942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c89f8-c8f0-4c87-bf6c-2bd521b241a3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:30.905270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c89f8-c8f0-4c87-bf6c-2bd521b241a3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:30.905477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c89f8-c8f0-4c87-bf6c-2bd521b241a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:30.925548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c40c89f8-c8f0-4c87-bf6c-2bd521b241a3 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:30.926794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c40c89f8-c8f0-4c87-bf6c-2bd521b241a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:30.982608 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:31.000039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 157/312] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:31.273445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e7e41-3005-4a7a-bade-4aecda43e463 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:31.381292 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:31.443474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e7e41-3005-4a7a-bade-4aecda43e463 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:31.484895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab864e6a-378c-4ab2-abdf-bfc0bd2fcc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:31.485930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab864e6a-378c-4ab2-abdf-bfc0bd2fcc46 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:31.486456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab864e6a-378c-4ab2-abdf-bfc0bd2fcc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:31.486927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab864e6a-378c-4ab2-abdf-bfc0bd2fcc46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:31.499399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab864e6a-378c-4ab2-abdf-bfc0bd2fcc46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ab864e6a-378c-4ab2-abdf-bfc0bd2fcc46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:31.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:31.506259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:31.507040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:31.507040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:31.507040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:31.507040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:31.507040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:31.507040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:31.507040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:31.507040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:31.507040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:31.507040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:31.507040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:31.507040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:31.509854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 156/313] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:31.519219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c89f8-c8f0-4c87-bf6c-2bd521b241a3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:31.709360 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c89f8-c8f0-4c87-bf6c-2bd521b241a3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:31.783118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9a3ad-a3cb-4f20-95ae-303f37ec45da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:31.784179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9a3ad-a3cb-4f20-95ae-303f37ec45da None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:31.785250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9a3ad-a3cb-4f20-95ae-303f37ec45da None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:31.785728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9a3ad-a3cb-4f20-95ae-303f37ec45da None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:31.787049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9a3ad-a3cb-4f20-95ae-303f37ec45da None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:31.787590 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9a3ad-a3cb-4f20-95ae-303f37ec45da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:31.790314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9a3ad-a3cb-4f20-95ae-303f37ec45da None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:31.807777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 158/314] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:31.817301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab864e6a-378c-4ab2-abdf-bfc0bd2fcc46 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:31.980755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab864e6a-378c-4ab2-abdf-bfc0bd2fcc46 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:32.009381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:32.033233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad2a2c-837e-4cb1-b873-f7ca300ec320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:32.033900 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad2a2c-837e-4cb1-b873-f7ca300ec320 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:32.034285 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad2a2c-837e-4cb1-b873-f7ca300ec320 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:32.035109 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad2a2c-837e-4cb1-b873-f7ca300ec320 None admin] PATH_INFO: `/v3/users/83cd3ce4c3e449eaa1d7ffe84a33b96c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:32.036437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad2a2c-837e-4cb1-b873-f7ca300ec320 None admin] RBAC: Authorizing `identity:get_user(user_id=83cd3ce4c3e449eaa1d7ffe84a33b96c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:32.038131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad2a2c-837e-4cb1-b873-f7ca300ec320 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:32.040381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 157/315] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:31 2022] GET /identity/v3/users/83cd3ce4c3e449eaa1d7ffe84a33b96c => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:32.048945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9a3ad-a3cb-4f20-95ae-303f37ec45da None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:32.126825 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77917e-6548-4ebf-a755-1575eebca525 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:32.127243 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77917e-6548-4ebf-a755-1575eebca525 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:32.127525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77917e-6548-4ebf-a755-1575eebca525 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:32.129172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 158/316] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:32.135037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4340592-db7f-42c5-bf79-d80f37efeeac None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:32.135408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4340592-db7f-42c5-bf79-d80f37efeeac None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:32.135636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4340592-db7f-42c5-bf79-d80f37efeeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:32.141040 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:32.156961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d4340592-db7f-42c5-bf79-d80f37efeeac None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:32.158224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d4340592-db7f-42c5-bf79-d80f37efeeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:32.246448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 159/317] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:32.247767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9a3ad-a3cb-4f20-95ae-303f37ec45da None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:32.311718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba8e4c-8ea3-4602-9fa5-653b9d7dbd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:32.313034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba8e4c-8ea3-4602-9fa5-653b9d7dbd37 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:32.313606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba8e4c-8ea3-4602-9fa5-653b9d7dbd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:32.314089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba8e4c-8ea3-4602-9fa5-653b9d7dbd37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:32.442864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba8e4c-8ea3-4602-9fa5-653b9d7dbd37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:32.446060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba8e4c-8ea3-4602-9fa5-653b9d7dbd37 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:32.525360 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4340592-db7f-42c5-bf79-d80f37efeeac None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:32.582416 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:32.697300 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 159/318] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:32.704284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba8e4c-8ea3-4602-9fa5-653b9d7dbd37 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:32.768363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4340592-db7f-42c5-bf79-d80f37efeeac None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:32.797412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:32.828485 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba8e4c-8ea3-4602-9fa5-653b9d7dbd37 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:32.837638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638e42e-f411-44ab-8e63-d638abf22259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:32.838712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638e42e-f411-44ab-8e63-d638abf22259 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:32.839674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638e42e-f411-44ab-8e63-d638abf22259 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:32.840467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638e42e-f411-44ab-8e63-d638abf22259 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:32.843784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638e42e-f411-44ab-8e63-d638abf22259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:32.846600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638e42e-f411-44ab-8e63-d638abf22259 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:32.849609 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 160/319] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:25:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:32.851658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e83e72-a900-4824-9c97-02699ef1a038 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:32.852136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e83e72-a900-4824-9c97-02699ef1a038 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:32.852615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e83e72-a900-4824-9c97-02699ef1a038 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:32.855643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 161/320] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:32.858508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e83e72-a900-4824-9c97-02699ef1a038 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:32.883755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5561d-db15-444f-a6ef-69f9275dc498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:32.884502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5561d-db15-444f-a6ef-69f9275dc498 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:32.884777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5561d-db15-444f-a6ef-69f9275dc498 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:32.885046 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5561d-db15-444f-a6ef-69f9275dc498 None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:32.886444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5561d-db15-444f-a6ef-69f9275dc498 None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:32.888124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5561d-db15-444f-a6ef-69f9275dc498 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:32.890393 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 160/321] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:32 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:32.897225 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b06c1-e6f6-40bd-b4c7-dff96f1d0033 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:32.897477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b06c1-e6f6-40bd-b4c7-dff96f1d0033 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:32.897807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b06c1-e6f6-40bd-b4c7-dff96f1d0033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:32.918567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ca2b06c1-e6f6-40bd-b4c7-dff96f1d0033 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:32.919790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ca2b06c1-e6f6-40bd-b4c7-dff96f1d0033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:32.931065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:33.006737 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 161/322] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:33.011410 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b06c1-e6f6-40bd-b4c7-dff96f1d0033 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:33.069687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e83e72-a900-4824-9c97-02699ef1a038 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:33.132650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdff6c-61aa-4a03-8359-0a90e17f7f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:33.133618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdff6c-61aa-4a03-8359-0a90e17f7f86 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:33.134049 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdff6c-61aa-4a03-8359-0a90e17f7f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:33.134498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdff6c-61aa-4a03-8359-0a90e17f7f86 None admin] PATH_INFO: `/v3/groups` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:33.135401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdff6c-61aa-4a03-8359-0a90e17f7f86 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:33.137038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdff6c-61aa-4a03-8359-0a90e17f7f86 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:33.205507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b06c1-e6f6-40bd-b4c7-dff96f1d0033 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:33.249905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 162/323] 10.222.0.44 () {60 vars in 1134 bytes} [Sun Sep 11 08:25:32 2022] POST /identity/v3/groups => generated 226 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:25:33.253524 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381454af-ec21-4672-b456-11e2227b135f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:33.254084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381454af-ec21-4672-b456-11e2227b135f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:33.254700 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381454af-ec21-4672-b456-11e2227b135f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:33.262225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 163/324] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:33.267206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ed7ab-4526-4856-b521-9ae84739831f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:33.268172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ed7ab-4526-4856-b521-9ae84739831f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:33.269775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ed7ab-4526-4856-b521-9ae84739831f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:33.271266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125db309-a98e-428b-8d01-9f0b079ed598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:33.272275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125db309-a98e-428b-8d01-9f0b079ed598 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:33.272982 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125db309-a98e-428b-8d01-9f0b079ed598 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:33.273462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125db309-a98e-428b-8d01-9f0b079ed598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:33.274998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125db309-a98e-428b-8d01-9f0b079ed598 None admin] RBAC: Adding query filter params (role.id=2684d96517574aa7bb00050a3c0e2d92, scope.project.id=ac74b750ef33436a83f5ee0474a71920, user.id=83cd3ce4c3e449eaa1d7ffe84a33b96c) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:33.275678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125db309-a98e-428b-8d01-9f0b079ed598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:33.278874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125db309-a98e-428b-8d01-9f0b079ed598 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:33.292577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 162/325] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:25:33 2022] GET /identity/v3/role_assignments?user.id=83cd3ce4c3e449eaa1d7ffe84a33b96c&scope.project.id=ac74b750ef33436a83f5ee0474a71920&role.id=2684d96517574aa7bb00050a3c0e2d92 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:33.295724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125db309-a98e-428b-8d01-9f0b079ed598 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:33.306593 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:33.317868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a31ed7ab-4526-4856-b521-9ae84739831f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:33.322251 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a31ed7ab-4526-4856-b521-9ae84739831f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:33.409864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:33.447349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125db309-a98e-428b-8d01-9f0b079ed598 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:33.456747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 164/326] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:33.506265 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e8a3e-9bb0-4cbc-8e57-2babc6bf7422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:33.506927 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e8a3e-9bb0-4cbc-8e57-2babc6bf7422 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:33.507205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e8a3e-9bb0-4cbc-8e57-2babc6bf7422 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:33.507477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e8a3e-9bb0-4cbc-8e57-2babc6bf7422 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:33.521880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e8a3e-9bb0-4cbc-8e57-2babc6bf7422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-873e8a3e-9bb0-4cbc-8e57-2babc6bf7422 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:33.527907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:33.530316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:33.531714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:33.531714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:33.531714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:33.531714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:33.531714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:33.531714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:33.531714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:33.531714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:33.531714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:33.531714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:33.531714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:33.531714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:33.531714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 163/327] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:25:33.538714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ed7ab-4526-4856-b521-9ae84739831f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:33.719610 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ed7ab-4526-4856-b521-9ae84739831f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:33.782061 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0c5f8-38ce-41e5-be7a-782aadc81450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:33.783663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0c5f8-38ce-41e5-be7a-782aadc81450 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:33.784507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0c5f8-38ce-41e5-be7a-782aadc81450 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:33.785261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0c5f8-38ce-41e5-be7a-782aadc81450 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:33.786964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0c5f8-38ce-41e5-be7a-782aadc81450 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:33.787718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0c5f8-38ce-41e5-be7a-782aadc81450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:33.790433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0c5f8-38ce-41e5-be7a-782aadc81450 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:33.808285 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 165/328] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:33.813650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e8a3e-9bb0-4cbc-8e57-2babc6bf7422 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:33.906385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:33.964785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e8a3e-9bb0-4cbc-8e57-2babc6bf7422 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:33.981966 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0c5f8-38ce-41e5-be7a-782aadc81450 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:34.017217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd4885-aff8-47c3-ab0b-25070a9733d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:34.017870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd4885-aff8-47c3-ab0b-25070a9733d1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:34.018165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd4885-aff8-47c3-ab0b-25070a9733d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:34.018482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd4885-aff8-47c3-ab0b-25070a9733d1 None admin] PATH_INFO: `/v3/users/7e8931f249d049e38e860f7ab6bbffb6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:34.019864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd4885-aff8-47c3-ab0b-25070a9733d1 None admin] RBAC: Authorizing `identity:get_user(user_id=7e8931f249d049e38e860f7ab6bbffb6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:34.021568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd4885-aff8-47c3-ab0b-25070a9733d1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:34.023355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 164/329] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:33 2022] GET /identity/v3/users/7e8931f249d049e38e860f7ab6bbffb6 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:34.036110 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd4885-aff8-47c3-ab0b-25070a9733d1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:34.041095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:34.135882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0c5f8-38ce-41e5-be7a-782aadc81450 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:34.180300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ca116-757a-4a76-b4eb-61cd97ace318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:34.181389 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ca116-757a-4a76-b4eb-61cd97ace318 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:34.181884 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ca116-757a-4a76-b4eb-61cd97ace318 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:34.182661 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ca116-757a-4a76-b4eb-61cd97ace318 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:34.186662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd4885-aff8-47c3-ab0b-25070a9733d1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:34.213066 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ca116-757a-4a76-b4eb-61cd97ace318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:34.226818 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d5aae-d160-4e90-aa76-97c80e425d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7d4ca116-757a-4a76-b4eb-61cd97ace318 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:34.227338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:34.228169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:34.228952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:34.228952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:34.228952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:34.228952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:34.228952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:34.228952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:34.228952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:34.228952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:34.228952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:34.228952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:34.228952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:34.228952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:34.229551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d5aae-d160-4e90-aa76-97c80e425d47 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:34.229551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d5aae-d160-4e90-aa76-97c80e425d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:34.229551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d5aae-d160-4e90-aa76-97c80e425d47 None admin] PATH_INFO: `/v3/system/users/7e8931f249d049e38e860f7ab6bbffb6/roles/2684d96517574aa7bb00050a3c0e2d92` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:34.229551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d5aae-d160-4e90-aa76-97c80e425d47 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7e8931f249d049e38e860f7ab6bbffb6, role_id=2684d96517574aa7bb00050a3c0e2d92)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:34.229803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 166/330] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:34.234270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d5aae-d160-4e90-aa76-97c80e425d47 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:34.234270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:34.239002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ca116-757a-4a76-b4eb-61cd97ace318 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:34.317060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 165/331] 10.222.0.44 () {58 vars in 1334 bytes} [Sun Sep 11 08:25:34 2022] PUT /identity/v3/system/users/7e8931f249d049e38e860f7ab6bbffb6/roles/2684d96517574aa7bb00050a3c0e2d92 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:25:34.370865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:34.379689 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ca116-757a-4a76-b4eb-61cd97ace318 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:34.424627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5b2e0-c54b-4ace-8207-4815f8ad561e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:34.425836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5b2e0-c54b-4ace-8207-4815f8ad561e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:34.426268 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5b2e0-c54b-4ace-8207-4815f8ad561e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:34.426545 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5b2e0-c54b-4ace-8207-4815f8ad561e None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:34.427241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5b2e0-c54b-4ace-8207-4815f8ad561e None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:34.427538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5b2e0-c54b-4ace-8207-4815f8ad561e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:34.429021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5b2e0-c54b-4ace-8207-4815f8ad561e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:34.440320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 167/332] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:34.448965 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d5aae-d160-4e90-aa76-97c80e425d47 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:34.636083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d5aae-d160-4e90-aa76-97c80e425d47 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:34.716646 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc45bc-a1cc-4cc6-8d26-bce57d8c5728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:34.717693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc45bc-a1cc-4cc6-8d26-bce57d8c5728 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:34.718186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc45bc-a1cc-4cc6-8d26-bce57d8c5728 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:34.718668 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc45bc-a1cc-4cc6-8d26-bce57d8c5728 None admin] PATH_INFO: `/v3/users/d5e45f5cc6354082af5b120c87b64d61` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:34.721439 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc45bc-a1cc-4cc6-8d26-bce57d8c5728 None admin] RBAC: Authorizing `identity:get_user(user_id=d5e45f5cc6354082af5b120c87b64d61)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:34.724798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc45bc-a1cc-4cc6-8d26-bce57d8c5728 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:34.728042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 166/333] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:34 2022] GET /identity/v3/users/d5e45f5cc6354082af5b120c87b64d61 => generated 293 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:34.737044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5b2e0-c54b-4ace-8207-4815f8ad561e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:34.909982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5b2e0-c54b-4ace-8207-4815f8ad561e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:34.953652 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7712065-dffe-4882-9fee-d3e1669f3bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:34.954318 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7712065-dffe-4882-9fee-d3e1669f3bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:34.954645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7712065-dffe-4882-9fee-d3e1669f3bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:34.954918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7712065-dffe-4882-9fee-d3e1669f3bd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:34.955654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7712065-dffe-4882-9fee-d3e1669f3bd1 None admin] RBAC: Adding query filter params (role.id=dba75ce7d709483fbbb0027d29e99725, user.id=d5e45f5cc6354082af5b120c87b64d61, scope.system=all) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:34.956026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7712065-dffe-4882-9fee-d3e1669f3bd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:34.958270 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7712065-dffe-4882-9fee-d3e1669f3bd1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:34.973022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 168/334] 10.222.0.44 () {56 vars in 1308 bytes} [Sun Sep 11 08:25:34 2022] GET /identity/v3/role_assignments?user.id=d5e45f5cc6354082af5b120c87b64d61&scope.system=all&role.id=dba75ce7d709483fbbb0027d29e99725 => generated 510 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:36.110538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb52b18-f27a-4dcb-a4c2-41b5152cc14f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:36.111258 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb52b18-f27a-4dcb-a4c2-41b5152cc14f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:36.111431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb52b18-f27a-4dcb-a4c2-41b5152cc14f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:36.114273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 167/335] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:36.120171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a2f2c-fcba-4d3b-95a0-8865ad9fba20 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:36.120803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a2f2c-fcba-4d3b-95a0-8865ad9fba20 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:36.120803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a2f2c-fcba-4d3b-95a0-8865ad9fba20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:36.147328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7a0a2f2c-fcba-4d3b-95a0-8865ad9fba20 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:36.148460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7a0a2f2c-fcba-4d3b-95a0-8865ad9fba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:36.160542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:36.247577 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 169/336] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:36.539053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb52b18-f27a-4dcb-a4c2-41b5152cc14f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:36.628734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:36.671476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb52b18-f27a-4dcb-a4c2-41b5152cc14f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:36.715673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d111a-712c-4970-a54a-8e239aa6c605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:36.716346 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d111a-712c-4970-a54a-8e239aa6c605 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:36.716625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d111a-712c-4970-a54a-8e239aa6c605 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:36.716887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d111a-712c-4970-a54a-8e239aa6c605 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:36.733418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d111a-712c-4970-a54a-8e239aa6c605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-234d111a-712c-4970-a54a-8e239aa6c605 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:36.739264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:36.741016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:36.743280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:36.743280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:36.743280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:36.743280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:36.743280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:36.743280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:36.743280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:36.743280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:36.743280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:36.743280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:36.743280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:36.743280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:36.743280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 168/337] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:25:36.752449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a2f2c-fcba-4d3b-95a0-8865ad9fba20 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:37.012988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a2f2c-fcba-4d3b-95a0-8865ad9fba20 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:37.095867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00b4e7-d0a7-4300-958c-a221cd57ff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:37.096985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00b4e7-d0a7-4300-958c-a221cd57ff1b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:37.097456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00b4e7-d0a7-4300-958c-a221cd57ff1b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:37.097939 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00b4e7-d0a7-4300-958c-a221cd57ff1b None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:37.099234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00b4e7-d0a7-4300-958c-a221cd57ff1b None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:37.099781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00b4e7-d0a7-4300-958c-a221cd57ff1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:37.102456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00b4e7-d0a7-4300-958c-a221cd57ff1b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:37.126009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 170/338] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:37.132840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d111a-712c-4970-a54a-8e239aa6c605 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:37.183990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349afee4-3262-48ec-92ce-af5281fdc202 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:37.184479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349afee4-3262-48ec-92ce-af5281fdc202 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:37.184954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349afee4-3262-48ec-92ce-af5281fdc202 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:37.187778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 171/339] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:37.202715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e190587-400b-47ea-8226-b44596fb2913 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:37.203138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e190587-400b-47ea-8226-b44596fb2913 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:37.203566 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e190587-400b-47ea-8226-b44596fb2913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:37.239575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:37.258503 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6e190587-400b-47ea-8226-b44596fb2913 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:37.262674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6e190587-400b-47ea-8226-b44596fb2913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:37.384711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d111a-712c-4970-a54a-8e239aa6c605 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:37.449384 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:37.467504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f3373-7f1c-457a-ab5e-914f1e3686da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:37.468146 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f3373-7f1c-457a-ab5e-914f1e3686da None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:37.468425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f3373-7f1c-457a-ab5e-914f1e3686da None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:37.468696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f3373-7f1c-457a-ab5e-914f1e3686da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:37.498323 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 172/340] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:37.579320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f3373-7f1c-457a-ab5e-914f1e3686da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:37.580820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f3373-7f1c-457a-ab5e-914f1e3686da None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:37.726418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 169/341] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:37.733033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e190587-400b-47ea-8226-b44596fb2913 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:37.865803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e190587-400b-47ea-8226-b44596fb2913 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:37.911017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c382c3-87bc-41f3-92e0-6d9a56c3d696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:37.911708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c382c3-87bc-41f3-92e0-6d9a56c3d696 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:37.911983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c382c3-87bc-41f3-92e0-6d9a56c3d696 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:37.912253 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c382c3-87bc-41f3-92e0-6d9a56c3d696 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:37.913611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c382c3-87bc-41f3-92e0-6d9a56c3d696 None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:37.915172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c382c3-87bc-41f3-92e0-6d9a56c3d696 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:37.917516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 173/342] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:37 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:37.923356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f3373-7f1c-457a-ab5e-914f1e3686da None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:37.983915 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:38.083248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8cf41-125d-4268-905f-98aa218db939 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:38.083712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8cf41-125d-4268-905f-98aa218db939 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:38.084324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8cf41-125d-4268-905f-98aa218db939 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:38.089212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 174/343] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:38.101654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41861b8f-6a3f-4faa-b48e-4ff62adddb67 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:38.102137 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41861b8f-6a3f-4faa-b48e-4ff62adddb67 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:38.102656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41861b8f-6a3f-4faa-b48e-4ff62adddb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:38.115394 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f3373-7f1c-457a-ab5e-914f1e3686da None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:38.136125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:38.166367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8983c2-81ff-47c9-82b9-f48b15e5efe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:38.166367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8983c2-81ff-47c9-82b9-f48b15e5efe7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:38.166367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8983c2-81ff-47c9-82b9-f48b15e5efe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:38.166367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8983c2-81ff-47c9-82b9-f48b15e5efe7 None admin] PATH_INFO: `/v3/groups/33a5b466ae1f4cd9a20f176b800520cc` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:38.167127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-41861b8f-6a3f-4faa-b48e-4ff62adddb67 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:38.168836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-41861b8f-6a3f-4faa-b48e-4ff62adddb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:38.191400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8983c2-81ff-47c9-82b9-f48b15e5efe7 None admin] RBAC: Authorizing `identity:get_group(group_id=33a5b466ae1f4cd9a20f176b800520cc)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:38.194254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8983c2-81ff-47c9-82b9-f48b15e5efe7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:38.197454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 170/344] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:37 2022] GET /identity/v3/groups/33a5b466ae1f4cd9a20f176b800520cc => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:38.203470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8983c2-81ff-47c9-82b9-f48b15e5efe7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:38.334194 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 175/345] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:38.335423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41861b8f-6a3f-4faa-b48e-4ff62adddb67 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:38.364390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:38.377066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8983c2-81ff-47c9-82b9-f48b15e5efe7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:38.433447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fde3a-ade0-4102-ab0a-5ebf3ebed14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:38.434500 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fde3a-ade0-4102-ab0a-5ebf3ebed14a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:38.434970 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fde3a-ade0-4102-ab0a-5ebf3ebed14a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:38.435419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fde3a-ade0-4102-ab0a-5ebf3ebed14a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:38.453069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fde3a-ade0-4102-ab0a-5ebf3ebed14a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b73fde3a-ade0-4102-ab0a-5ebf3ebed14a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:38.459867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:38.460794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:38.461651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:38.461651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:38.461651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:38.461651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:38.461651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:38.461651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:38.461651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:38.461651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:38.461651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:38.461651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:38.461651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:38.461651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:38.468641 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 171/346] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:25:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:38.483564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fde3a-ade0-4102-ab0a-5ebf3ebed14a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:38.547408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41861b8f-6a3f-4faa-b48e-4ff62adddb67 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:38.629165 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ec750-2414-4e2b-a958-522385a8e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:38.629165 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ec750-2414-4e2b-a958-522385a8e5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:38.629165 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ec750-2414-4e2b-a958-522385a8e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:38.629165 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ec750-2414-4e2b-a958-522385a8e5e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:38.629875 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ec750-2414-4e2b-a958-522385a8e5e4 None admin] RBAC: Adding query filter params (group.id=33a5b466ae1f4cd9a20f176b800520cc, role.id=dba75ce7d709483fbbb0027d29e99725, scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:38.630282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ec750-2414-4e2b-a958-522385a8e5e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:38.631896 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ec750-2414-4e2b-a958-522385a8e5e4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:38.638719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:38.640238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 176/347] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 08:25:38 2022] GET /identity/v3/role_assignments?group.id=33a5b466ae1f4cd9a20f176b800520cc&scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7&role.id=dba75ce7d709483fbbb0027d29e99725 => generated 260 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:38.652351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fde3a-ade0-4102-ab0a-5ebf3ebed14a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:38.702065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78294de1-6c08-4ac4-8876-6bce74f9095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:38.703139 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78294de1-6c08-4ac4-8876-6bce74f9095c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:38.703618 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78294de1-6c08-4ac4-8876-6bce74f9095c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:38.704101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78294de1-6c08-4ac4-8876-6bce74f9095c None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:38.705021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78294de1-6c08-4ac4-8876-6bce74f9095c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:38.705540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78294de1-6c08-4ac4-8876-6bce74f9095c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:38.707848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78294de1-6c08-4ac4-8876-6bce74f9095c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:38.722781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 172/348] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:25:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:38.728978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ec750-2414-4e2b-a958-522385a8e5e4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:38.943794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ec750-2414-4e2b-a958-522385a8e5e4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:39.001709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bbdab-3665-483b-8e4d-7cd0c7974fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:39.002438 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bbdab-3665-483b-8e4d-7cd0c7974fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:39.002718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bbdab-3665-483b-8e4d-7cd0c7974fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:39.003637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bbdab-3665-483b-8e4d-7cd0c7974fa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:39.093360 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78294de1-6c08-4ac4-8876-6bce74f9095c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:39.132831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:39.143785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:39.146308 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bbdab-3665-483b-8e4d-7cd0c7974fa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:39.147870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bbdab-3665-483b-8e4d-7cd0c7974fa8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:39.328971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78294de1-6c08-4ac4-8876-6bce74f9095c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:39.391928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 177/349] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:39.399868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f3fbe-f5ba-4970-8c1f-4ed84839faf1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:39.400180 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f3fbe-f5ba-4970-8c1f-4ed84839faf1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:39.400425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f3fbe-f5ba-4970-8c1f-4ed84839faf1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:39.401860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 178/350] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:39.402758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f3fbe-f5ba-4970-8c1f-4ed84839faf1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:39.431150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b8a6d-c021-429e-a6da-31fe37834c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:39.432399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b8a6d-c021-429e-a6da-31fe37834c25 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:39.432837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b8a6d-c021-429e-a6da-31fe37834c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:39.433334 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b8a6d-c021-429e-a6da-31fe37834c25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:39.458216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b8a6d-c021-429e-a6da-31fe37834c25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d86b8a6d-c021-429e-a6da-31fe37834c25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:39.466112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:39.467889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 173/351] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:39.469411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8860e9-fbb8-40e4-824b-7b86f2c660cf None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:39.472406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8860e9-fbb8-40e4-824b-7b86f2c660cf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:39.472406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8860e9-fbb8-40e4-824b-7b86f2c660cf None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:39.472406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 174/352] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:39.472406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f751aa6-5f96-4b2a-bd73-d95c38668be1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:39.472944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f751aa6-5f96-4b2a-bd73-d95c38668be1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:39.472944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f751aa6-5f96-4b2a-bd73-d95c38668be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:39.495406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9f751aa6-5f96-4b2a-bd73-d95c38668be1 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:39.496654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9f751aa6-5f96-4b2a-bd73-d95c38668be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:39.541114 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:39.548542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f3fbe-f5ba-4970-8c1f-4ed84839faf1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:39.600592 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 175/353] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:39.603664 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185dd623-2cf3-4d7d-97e9-ee489c899579 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:39.604473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185dd623-2cf3-4d7d-97e9-ee489c899579 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:39.605815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185dd623-2cf3-4d7d-97e9-ee489c899579 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:39.609292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 176/354] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:39.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366b18c-85e4-49e3-996b-5ab1b0d24d9d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:39.618371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366b18c-85e4-49e3-996b-5ab1b0d24d9d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:39.619061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366b18c-85e4-49e3-996b-5ab1b0d24d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:39.620872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8be1df-68aa-4fbd-930b-00af037772eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:39.623499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8be1df-68aa-4fbd-930b-00af037772eb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:39.623499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8be1df-68aa-4fbd-930b-00af037772eb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:39.623499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8be1df-68aa-4fbd-930b-00af037772eb None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:39.625765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8be1df-68aa-4fbd-930b-00af037772eb None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:39.628737 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8be1df-68aa-4fbd-930b-00af037772eb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:39.632484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 179/355] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:39 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:39.635838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8be1df-68aa-4fbd-930b-00af037772eb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:39.652425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:39.657195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8366b18c-85e4-49e3-996b-5ab1b0d24d9d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:39.659020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8366b18c-85e4-49e3-996b-5ab1b0d24d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:39.828650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 177/356] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:39.831786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3bf25-a768-4e5d-ba7b-646b87877200 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:39.832306 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3bf25-a768-4e5d-ba7b-646b87877200 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:39.832784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3bf25-a768-4e5d-ba7b-646b87877200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:39.883958 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-68b3bf25-a768-4e5d-ba7b-646b87877200 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:39.887035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-68b3bf25-a768-4e5d-ba7b-646b87877200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:39.899180 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8be1df-68aa-4fbd-930b-00af037772eb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:40.019886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839218a-4ac0-4686-9416-c84653a23805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:40.020978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839218a-4ac0-4686-9416-c84653a23805 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:40.026196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839218a-4ac0-4686-9416-c84653a23805 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:40.026196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839218a-4ac0-4686-9416-c84653a23805 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:40.026196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839218a-4ac0-4686-9416-c84653a23805 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:40.026196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839218a-4ac0-4686-9416-c84653a23805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:40.029039 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839218a-4ac0-4686-9416-c84653a23805 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:40.068678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 180/357] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:40.071921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839218a-4ac0-4686-9416-c84653a23805 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:40.092498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 178/358] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:40.098652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3bf25-a768-4e5d-ba7b-646b87877200 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:40.169312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:40.226745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:40.267978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3bf25-a768-4e5d-ba7b-646b87877200 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:40.311361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6530959-0ca6-482b-b2e2-d45b9c6f75db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:40.312316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6530959-0ca6-482b-b2e2-d45b9c6f75db None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:40.312674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6530959-0ca6-482b-b2e2-d45b9c6f75db None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:40.313046 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6530959-0ca6-482b-b2e2-d45b9c6f75db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:40.320067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839218a-4ac0-4686-9416-c84653a23805 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:40.327259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6530959-0ca6-482b-b2e2-d45b9c6f75db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c6530959-0ca6-482b-b2e2-d45b9c6f75db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:40.333279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:40.335765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 179/359] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:25:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:40.337476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6530959-0ca6-482b-b2e2-d45b9c6f75db None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:40.412672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547f2bc-e521-4638-abd4-86dee25f9270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:40.414209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547f2bc-e521-4638-abd4-86dee25f9270 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:40.414974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547f2bc-e521-4638-abd4-86dee25f9270 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:40.415670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547f2bc-e521-4638-abd4-86dee25f9270 None admin] PATH_INFO: `/v3/groups/33a5b466ae1f4cd9a20f176b800520cc` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:40.419040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547f2bc-e521-4638-abd4-86dee25f9270 None admin] RBAC: Authorizing `identity:get_group(group_id=33a5b466ae1f4cd9a20f176b800520cc)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:40.422099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547f2bc-e521-4638-abd4-86dee25f9270 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:40.426033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 181/360] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:40 2022] GET /identity/v3/groups/33a5b466ae1f4cd9a20f176b800520cc => generated 226 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:40.427955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547f2bc-e521-4638-abd4-86dee25f9270 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:40.468496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6530959-0ca6-482b-b2e2-d45b9c6f75db None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:40.508858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:40.514298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fea043-3cd4-4e79-9cbe-ea51606035e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:40.514918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fea043-3cd4-4e79-9cbe-ea51606035e6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:40.515200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fea043-3cd4-4e79-9cbe-ea51606035e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:40.515475 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fea043-3cd4-4e79-9cbe-ea51606035e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:40.602418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fea043-3cd4-4e79-9cbe-ea51606035e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:40.604040 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fea043-3cd4-4e79-9cbe-ea51606035e6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:40.620531 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:40.630457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547f2bc-e521-4638-abd4-86dee25f9270 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:40.680452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ad97f-32f3-49fa-8580-55422d18494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:40.681839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ad97f-32f3-49fa-8580-55422d18494a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:40.682493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ad97f-32f3-49fa-8580-55422d18494a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:40.683446 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ad97f-32f3-49fa-8580-55422d18494a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:40.704353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ad97f-32f3-49fa-8580-55422d18494a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-376ad97f-32f3-49fa-8580-55422d18494a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:40.712820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:40.713978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:40.715271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:40.715271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:40.715271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:40.715271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:40.715271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:40.715271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:40.715271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:40.715271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:40.715271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:40.715271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:40.715271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:40.715271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:40.717803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 182/361] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:25:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:40.718973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ad97f-32f3-49fa-8580-55422d18494a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:40.777809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 180/362] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:40.779162 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fea043-3cd4-4e79-9cbe-ea51606035e6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:40.820104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:40.896786 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ad97f-32f3-49fa-8580-55422d18494a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:40.957890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fea043-3cd4-4e79-9cbe-ea51606035e6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:40.967815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c89f2-d135-441d-8f97-912012b9ff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:40.968818 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c89f2-d135-441d-8f97-912012b9ff9a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:40.969371 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c89f2-d135-441d-8f97-912012b9ff9a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:40.969936 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c89f2-d135-441d-8f97-912012b9ff9a None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:40.970950 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c89f2-d135-441d-8f97-912012b9ff9a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:40.971477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c89f2-d135-441d-8f97-912012b9ff9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:40.973698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c89f2-d135-441d-8f97-912012b9ff9a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:40.992164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 183/363] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:25:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:40.992164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c89f2-d135-441d-8f97-912012b9ff9a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:41.018505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305dee2-a95e-467f-8afc-bfc18270e85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:41.019569 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305dee2-a95e-467f-8afc-bfc18270e85d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:41.019854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305dee2-a95e-467f-8afc-bfc18270e85d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:41.020124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305dee2-a95e-467f-8afc-bfc18270e85d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:41.020845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305dee2-a95e-467f-8afc-bfc18270e85d None admin] RBAC: Adding query filter params (group.id=33a5b466ae1f4cd9a20f176b800520cc, role.id=65f33fe3045c433b9a3e045e7572aed6, scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:41.021152 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305dee2-a95e-467f-8afc-bfc18270e85d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:41.022758 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305dee2-a95e-467f-8afc-bfc18270e85d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:41.026863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:41.029739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 181/364] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 08:25:40 2022] GET /identity/v3/role_assignments?group.id=33a5b466ae1f4cd9a20f176b800520cc&scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7&role.id=65f33fe3045c433b9a3e045e7572aed6 => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:41.030661 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305dee2-a95e-467f-8afc-bfc18270e85d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:41.224984 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:41.225633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c89f2-d135-441d-8f97-912012b9ff9a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:41.246698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305dee2-a95e-467f-8afc-bfc18270e85d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:41.291205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b9d1f-027e-4c8b-aabb-987e93657a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:41.291832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b9d1f-027e-4c8b-aabb-987e93657a49 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:41.292105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b9d1f-027e-4c8b-aabb-987e93657a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:41.292374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b9d1f-027e-4c8b-aabb-987e93657a49 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:41.293042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b9d1f-027e-4c8b-aabb-987e93657a49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:41.293344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b9d1f-027e-4c8b-aabb-987e93657a49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:41.295670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b9d1f-027e-4c8b-aabb-987e93657a49 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:41.305325 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 182/365] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:41.306273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b9d1f-027e-4c8b-aabb-987e93657a49 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:41.320652 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328461d1-bfd8-4f6b-8d5e-73420333a0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:41.322203 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328461d1-bfd8-4f6b-8d5e-73420333a0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:41.323155 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328461d1-bfd8-4f6b-8d5e-73420333a0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:41.324024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328461d1-bfd8-4f6b-8d5e-73420333a0a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:41.352733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328461d1-bfd8-4f6b-8d5e-73420333a0a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-328461d1-bfd8-4f6b-8d5e-73420333a0a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:41.368775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:41.370732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:41.372330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:41.372330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:41.372330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:41.372330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:41.372330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:41.372330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:41.372330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:41.372330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:41.372330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:41.372330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:41.372330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 08:25:41.372330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:41.378270 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 184/366] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:41.384363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328461d1-bfd8-4f6b-8d5e-73420333a0a7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:41.424926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b9d1f-027e-4c8b-aabb-987e93657a49 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:41.463595 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498223b7-99e2-403d-bebe-90f349ac206a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:41.464253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498223b7-99e2-403d-bebe-90f349ac206a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:41.464795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498223b7-99e2-403d-bebe-90f349ac206a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:41.465066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498223b7-99e2-403d-bebe-90f349ac206a None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:41.466348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498223b7-99e2-403d-bebe-90f349ac206a None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:41.467877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498223b7-99e2-403d-bebe-90f349ac206a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:41.469678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 183/367] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:41 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:41.470532 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498223b7-99e2-403d-bebe-90f349ac206a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:41.504032 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:41.560876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:41.619645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328461d1-bfd8-4f6b-8d5e-73420333a0a7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:41.679633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d3bbd-42a4-48f5-b283-2423e1a76d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:41.680388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d3bbd-42a4-48f5-b283-2423e1a76d40 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:41.680680 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d3bbd-42a4-48f5-b283-2423e1a76d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:41.680921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d3bbd-42a4-48f5-b283-2423e1a76d40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:41.700387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498223b7-99e2-403d-bebe-90f349ac206a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:41.751784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10f162-8b9a-489d-8069-f792ec1a2e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:41.753346 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10f162-8b9a-489d-8069-f792ec1a2e18 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:41.753829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10f162-8b9a-489d-8069-f792ec1a2e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:41.755322 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10f162-8b9a-489d-8069-f792ec1a2e18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:41.783119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4d3bbd-42a4-48f5-b283-2423e1a76d40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:41.787357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4d3bbd-42a4-48f5-b283-2423e1a76d40 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:41.897120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10f162-8b9a-489d-8069-f792ec1a2e18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:41.899430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10f162-8b9a-489d-8069-f792ec1a2e18 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:41.928855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:41.937953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:41.984507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 185/368] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:41.985627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d3bbd-42a4-48f5-b283-2423e1a76d40 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:42.105940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 184/369] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:42.107841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10f162-8b9a-489d-8069-f792ec1a2e18 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:42.142201 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d3bbd-42a4-48f5-b283-2423e1a76d40 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:42.215424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb283dd-0e70-412b-ae4e-d200589e7155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:42.216809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb283dd-0e70-412b-ae4e-d200589e7155 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:42.217445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb283dd-0e70-412b-ae4e-d200589e7155 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:42.217893 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb283dd-0e70-412b-ae4e-d200589e7155 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:42.219060 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb283dd-0e70-412b-ae4e-d200589e7155 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:42.220961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb283dd-0e70-412b-ae4e-d200589e7155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:42.223473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb283dd-0e70-412b-ae4e-d200589e7155 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:42.240132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 186/370] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:42.243235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49babda3-f657-4803-87f4-edc768a6df0e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:42.244499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49babda3-f657-4803-87f4-edc768a6df0e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:42.245015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49babda3-f657-4803-87f4-edc768a6df0e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:42.247587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 187/371] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:42.249001 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49babda3-f657-4803-87f4-edc768a6df0e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:42.281279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10f162-8b9a-489d-8069-f792ec1a2e18 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:42.310753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:42.327350 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:42.354737 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8a1f7-8da9-48d7-b9b9-0c97e4bd29ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:42.355870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8a1f7-8da9-48d7-b9b9-0c97e4bd29ca None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:42.355870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8a1f7-8da9-48d7-b9b9-0c97e4bd29ca None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:42.355870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8a1f7-8da9-48d7-b9b9-0c97e4bd29ca None admin] PATH_INFO: `/v3/groups/33a5b466ae1f4cd9a20f176b800520cc` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:42.357192 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8a1f7-8da9-48d7-b9b9-0c97e4bd29ca None admin] RBAC: Authorizing `identity:get_group(group_id=33a5b466ae1f4cd9a20f176b800520cc)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:42.359971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8a1f7-8da9-48d7-b9b9-0c97e4bd29ca None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:42.366351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 185/372] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:42 2022] GET /identity/v3/groups/33a5b466ae1f4cd9a20f176b800520cc => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:42.366351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8a1f7-8da9-48d7-b9b9-0c97e4bd29ca None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:42.428634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49babda3-f657-4803-87f4-edc768a6df0e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:42.473538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a65d1-b480-4de0-ace0-1a5331b520b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:42.474584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a65d1-b480-4de0-ace0-1a5331b520b6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:42.474985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a65d1-b480-4de0-ace0-1a5331b520b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:42.475396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a65d1-b480-4de0-ace0-1a5331b520b6 None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:42.476999 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a65d1-b480-4de0-ace0-1a5331b520b6 None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:42.479476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a65d1-b480-4de0-ace0-1a5331b520b6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:42.489126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 188/373] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:42 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:42.489126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a65d1-b480-4de0-ace0-1a5331b520b6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:42.534888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8a1f7-8da9-48d7-b9b9-0c97e4bd29ca None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:42.582425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a20fce-54f0-49ee-9255-537f3ae7e2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:42.583357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a20fce-54f0-49ee-9255-537f3ae7e2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:42.583773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a20fce-54f0-49ee-9255-537f3ae7e2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:42.584173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a20fce-54f0-49ee-9255-537f3ae7e2c6 None admin] PATH_INFO: `/v3/projects/878028e7aaff4c28815b9cd862f1c36e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:42.587040 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a20fce-54f0-49ee-9255-537f3ae7e2c6 None admin] RBAC: Authorizing `identity:get_project(project_id=878028e7aaff4c28815b9cd862f1c36e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:42.588815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a20fce-54f0-49ee-9255-537f3ae7e2c6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:42.591113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 186/374] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:42 2022] GET /identity/v3/projects/878028e7aaff4c28815b9cd862f1c36e => generated 344 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:42.594527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79169c9b-41a4-446b-88ed-6dc0d0de4e8d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:42.595009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79169c9b-41a4-446b-88ed-6dc0d0de4e8d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:42.595511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79169c9b-41a4-446b-88ed-6dc0d0de4e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:42.618487 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-79169c9b-41a4-446b-88ed-6dc0d0de4e8d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:42.619649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-79169c9b-41a4-446b-88ed-6dc0d0de4e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:42.630767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a65d1-b480-4de0-ace0-1a5331b520b6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:42.642283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:42.677302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4625e-32e4-42bc-a64b-297ba65a3abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:42.677976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4625e-32e4-42bc-a64b-297ba65a3abb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:42.678279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4625e-32e4-42bc-a64b-297ba65a3abb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:42.678554 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4625e-32e4-42bc-a64b-297ba65a3abb None admin] PATH_INFO: `/v3/users/7e8931f249d049e38e860f7ab6bbffb6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:42.679912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4625e-32e4-42bc-a64b-297ba65a3abb None admin] RBAC: Authorizing `identity:get_user(user_id=7e8931f249d049e38e860f7ab6bbffb6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:42.681493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4625e-32e4-42bc-a64b-297ba65a3abb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:42.693823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 189/375] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:25:42 2022] GET /identity/v3/users/7e8931f249d049e38e860f7ab6bbffb6 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:42.695037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:42.695544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4625e-32e4-42bc-a64b-297ba65a3abb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:42.723932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 187/376] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:42.724913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79169c9b-41a4-446b-88ed-6dc0d0de4e8d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:42.843726 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79169c9b-41a4-446b-88ed-6dc0d0de4e8d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:42.862159 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4625e-32e4-42bc-a64b-297ba65a3abb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:42.890686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffb33f-9c7f-4ada-97c9-f0a03afbcd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:42.891783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffb33f-9c7f-4ada-97c9-f0a03afbcd7a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:42.892236 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffb33f-9c7f-4ada-97c9-f0a03afbcd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:42.893089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffb33f-9c7f-4ada-97c9-f0a03afbcd7a None admin] PATH_INFO: `/v3/groups/33a5b466ae1f4cd9a20f176b800520cc` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:42.895596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffb33f-9c7f-4ada-97c9-f0a03afbcd7a None admin] RBAC: Authorizing `identity:get_group(group_id=33a5b466ae1f4cd9a20f176b800520cc)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:42.898306 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffb33f-9c7f-4ada-97c9-f0a03afbcd7a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:42.901116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 188/377] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:42 2022] GET /identity/v3/groups/33a5b466ae1f4cd9a20f176b800520cc => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:42.902179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffb33f-9c7f-4ada-97c9-f0a03afbcd7a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:42.911280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85078119-8d79-46c1-b11b-52e125bbff83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:42.912036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85078119-8d79-46c1-b11b-52e125bbff83 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:42.912386 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85078119-8d79-46c1-b11b-52e125bbff83 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:42.912730 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85078119-8d79-46c1-b11b-52e125bbff83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:42.913563 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85078119-8d79-46c1-b11b-52e125bbff83 None admin] RBAC: Adding query filter params (group.id=33a5b466ae1f4cd9a20f176b800520cc, role.id=dba75ce7d709483fbbb0027d29e99725, scope.project.id=ac74b750ef33436a83f5ee0474a71920) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:42.914504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85078119-8d79-46c1-b11b-52e125bbff83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:42.915573 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85078119-8d79-46c1-b11b-52e125bbff83 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:42.926487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 190/378] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 08:25:42 2022] GET /identity/v3/role_assignments?group.id=33a5b466ae1f4cd9a20f176b800520cc&scope.project.id=ac74b750ef33436a83f5ee0474a71920&role.id=dba75ce7d709483fbbb0027d29e99725 => generated 260 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:42.926487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85078119-8d79-46c1-b11b-52e125bbff83 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:43.014721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:43.020366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:43.099834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85078119-8d79-46c1-b11b-52e125bbff83 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:43.170291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffb33f-9c7f-4ada-97c9-f0a03afbcd7a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:43.176631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02489e85-9078-412d-9815-fa010bb90589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:43.177603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02489e85-9078-412d-9815-fa010bb90589 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:43.177879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02489e85-9078-412d-9815-fa010bb90589 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:43.178174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02489e85-9078-412d-9815-fa010bb90589 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:43.178908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02489e85-9078-412d-9815-fa010bb90589 None admin] RBAC: Adding query filter params (role.id=2684d96517574aa7bb00050a3c0e2d92, user.id=7e8931f249d049e38e860f7ab6bbffb6, scope.system=all) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:43.179215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02489e85-9078-412d-9815-fa010bb90589 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:43.180709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02489e85-9078-412d-9815-fa010bb90589 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:43.202256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 191/379] 10.222.0.44 () {56 vars in 1308 bytes} [Sun Sep 11 08:25:42 2022] GET /identity/v3/role_assignments?user.id=7e8931f249d049e38e860f7ab6bbffb6&scope.system=all&role.id=2684d96517574aa7bb00050a3c0e2d92 => generated 510 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:43.203503 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02489e85-9078-412d-9815-fa010bb90589 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:43.254450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1a2e8-5161-46ec-b55a-f4cf4c2f88a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:43.256326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1a2e8-5161-46ec-b55a-f4cf4c2f88a1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:43.258880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1a2e8-5161-46ec-b55a-f4cf4c2f88a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:43.262643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1a2e8-5161-46ec-b55a-f4cf4c2f88a1 None admin] PATH_INFO: `/v3/groups/8d6fc93d97044403ac1b6ea456cac2bf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:43.283722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1a2e8-5161-46ec-b55a-f4cf4c2f88a1 None admin] RBAC: Authorizing `identity:get_group(group_id=8d6fc93d97044403ac1b6ea456cac2bf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:43.291296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1a2e8-5161-46ec-b55a-f4cf4c2f88a1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:43.294519 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 189/380] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:42 2022] GET /identity/v3/groups/8d6fc93d97044403ac1b6ea456cac2bf => generated 229 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:43.307418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1a2e8-5161-46ec-b55a-f4cf4c2f88a1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:43.458471 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:43.466921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02489e85-9078-412d-9815-fa010bb90589 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:43.506779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1a2e8-5161-46ec-b55a-f4cf4c2f88a1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:43.538192 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:43.565939 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9402787-21ef-42bf-9000-5a404dcba480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:43.567115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9402787-21ef-42bf-9000-5a404dcba480 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:43.567613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9402787-21ef-42bf-9000-5a404dcba480 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:43.568084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9402787-21ef-42bf-9000-5a404dcba480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:43.568672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b84e9-8e59-438c-9bdf-facc62ab3249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:43.569267 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9402787-21ef-42bf-9000-5a404dcba480 None admin] RBAC: Adding query filter params (group.id=33a5b466ae1f4cd9a20f176b800520cc, role.id=65f33fe3045c433b9a3e045e7572aed6, scope.project.id=ac74b750ef33436a83f5ee0474a71920) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:43.569373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b84e9-8e59-438c-9bdf-facc62ab3249 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:43.569574 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b84e9-8e59-438c-9bdf-facc62ab3249 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:43.569842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b84e9-8e59-438c-9bdf-facc62ab3249 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:43.570266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9402787-21ef-42bf-9000-5a404dcba480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:43.571911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9402787-21ef-42bf-9000-5a404dcba480 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:43.580880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 192/381] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 08:25:43 2022] GET /identity/v3/role_assignments?group.id=33a5b466ae1f4cd9a20f176b800520cc&scope.project.id=ac74b750ef33436a83f5ee0474a71920&role.id=65f33fe3045c433b9a3e045e7572aed6 => generated 260 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:43.582239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9402787-21ef-42bf-9000-5a404dcba480 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:43.583648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b84e9-8e59-438c-9bdf-facc62ab3249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-982b84e9-8e59-438c-9bdf-facc62ab3249 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:43.590620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:43.591600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:43.597515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:43.597515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:43.597515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:43.597515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:43.597515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:43.597515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:43.597515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:43.597515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:43.597515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:43.597515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:43.597515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:43.597515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:43.597515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 190/382] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:43.600966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b84e9-8e59-438c-9bdf-facc62ab3249 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:43.751885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b84e9-8e59-438c-9bdf-facc62ab3249 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:43.758023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9402787-21ef-42bf-9000-5a404dcba480 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:43.793777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9e039-1a7f-4e00-b02e-9dfdfff03936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:43.794440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9e039-1a7f-4e00-b02e-9dfdfff03936 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:43.794715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9e039-1a7f-4e00-b02e-9dfdfff03936 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:43.794984 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9e039-1a7f-4e00-b02e-9dfdfff03936 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:43.795672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9e039-1a7f-4e00-b02e-9dfdfff03936 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:43.795973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9e039-1a7f-4e00-b02e-9dfdfff03936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:43.797404 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9e039-1a7f-4e00-b02e-9dfdfff03936 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:43.818613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 191/383] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:43.818613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4b8c3-e225-47ff-9f5b-a176e60171dc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:43.818613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4b8c3-e225-47ff-9f5b-a176e60171dc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:43.818613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4b8c3-e225-47ff-9f5b-a176e60171dc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:43.818613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 192/384] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:43.818613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1630e8-a5e6-4b37-ac88-7872ef543970 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:43.818613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1630e8-a5e6-4b37-ac88-7872ef543970 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:43.819719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1630e8-a5e6-4b37-ac88-7872ef543970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:43.828691 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf48bba-e0ae-4e57-b9ab-64ee296054d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:43.829718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf48bba-e0ae-4e57-b9ab-64ee296054d4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:43.830242 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf48bba-e0ae-4e57-b9ab-64ee296054d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:43.830718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf48bba-e0ae-4e57-b9ab-64ee296054d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:43.832252 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf48bba-e0ae-4e57-b9ab-64ee296054d4 None admin] RBAC: Adding query filter params (group.id=8d6fc93d97044403ac1b6ea456cac2bf, role.id=020bc02dca71485bab2312442adaa179, scope.project.id=878028e7aaff4c28815b9cd862f1c36e) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:43.832806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf48bba-e0ae-4e57-b9ab-64ee296054d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:43.835500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf48bba-e0ae-4e57-b9ab-64ee296054d4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:43.841413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fb1630e8-a5e6-4b37-ac88-7872ef543970 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:43.841696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fb1630e8-a5e6-4b37-ac88-7872ef543970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:43.848493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 193/385] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 08:25:43 2022] GET /identity/v3/role_assignments?group.id=8d6fc93d97044403ac1b6ea456cac2bf&scope.project.id=878028e7aaff4c28815b9cd862f1c36e&role.id=020bc02dca71485bab2312442adaa179 => generated 260 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:43.849771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf48bba-e0ae-4e57-b9ab-64ee296054d4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:43.889461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:43.933634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:43.938184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 193/386] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:44.092859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf48bba-e0ae-4e57-b9ab-64ee296054d4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:44.162561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ace6b-6e83-4374-a932-bc0ed863f7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:44.163552 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ace6b-6e83-4374-a932-bc0ed863f7fb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:44.163988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ace6b-6e83-4374-a932-bc0ed863f7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:44.164403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ace6b-6e83-4374-a932-bc0ed863f7fb None admin] PATH_INFO: `/v3/groups/33a5b466ae1f4cd9a20f176b800520cc` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:44.166213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ace6b-6e83-4374-a932-bc0ed863f7fb None admin] RBAC: Authorizing `identity:get_group(group_id=33a5b466ae1f4cd9a20f176b800520cc)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:44.168427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ace6b-6e83-4374-a932-bc0ed863f7fb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:44.171275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 194/387] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:43 2022] GET /identity/v3/groups/33a5b466ae1f4cd9a20f176b800520cc => generated 226 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:44.179373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1630e8-a5e6-4b37-ac88-7872ef543970 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:44.261387 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ace6b-6e83-4374-a932-bc0ed863f7fb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:44.334442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1630e8-a5e6-4b37-ac88-7872ef543970 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:44.406655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:44.417552 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ace6b-6e83-4374-a932-bc0ed863f7fb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:44.427452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc176df-b4a2-49ea-90ab-c89601ed6446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:44.428448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc176df-b4a2-49ea-90ab-c89601ed6446 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:44.428883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc176df-b4a2-49ea-90ab-c89601ed6446 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:44.429310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc176df-b4a2-49ea-90ab-c89601ed6446 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:44.432090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc176df-b4a2-49ea-90ab-c89601ed6446 None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:44.434973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc176df-b4a2-49ea-90ab-c89601ed6446 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:44.440877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 194/388] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:44 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:44.445781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc176df-b4a2-49ea-90ab-c89601ed6446 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:44.502020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935aac0-e098-486a-80e5-342084ecb6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:44.505496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935aac0-e098-486a-80e5-342084ecb6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:44.505496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935aac0-e098-486a-80e5-342084ecb6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:44.505496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935aac0-e098-486a-80e5-342084ecb6b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:44.520654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:44.537612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935aac0-e098-486a-80e5-342084ecb6b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a935aac0-e098-486a-80e5-342084ecb6b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:44.543230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:44.544174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:44.545000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:44.545000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:44.545000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:44.545000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:44.545000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:44.545000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:44.545000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:44.545000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:44.545000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:44.545000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:44.545000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:44.545000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:44.557255 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 195/389] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:25:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:44.561499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935aac0-e098-486a-80e5-342084ecb6b0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:44.668563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc176df-b4a2-49ea-90ab-c89601ed6446 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:44.740222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c351e-31c3-4913-9151-f530e04f9808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:44.740903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c351e-31c3-4913-9151-f530e04f9808 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:44.741176 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c351e-31c3-4913-9151-f530e04f9808 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:44.741443 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c351e-31c3-4913-9151-f530e04f9808 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7/groups/33a5b466ae1f4cd9a20f176b800520cc/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:44.744603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c351e-31c3-4913-9151-f530e04f9808 None admin] RBAC: Authorizing `identity:create_grant(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7, group_id=33a5b466ae1f4cd9a20f176b800520cc, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:44.746052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c351e-31c3-4913-9151-f530e04f9808 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:44.771765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935aac0-e098-486a-80e5-342084ecb6b0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:44.814100 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997fefb-e794-4b14-9269-dee58ae080b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:44.814955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997fefb-e794-4b14-9269-dee58ae080b5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:44.815777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997fefb-e794-4b14-9269-dee58ae080b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:44.816095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997fefb-e794-4b14-9269-dee58ae080b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:44.816983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997fefb-e794-4b14-9269-dee58ae080b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:44.817287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997fefb-e794-4b14-9269-dee58ae080b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:44.818960 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997fefb-e794-4b14-9269-dee58ae080b5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:44.825625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 195/390] 10.222.0.44 () {58 vars in 1442 bytes} [Sun Sep 11 08:25:44 2022] PUT /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7/groups/33a5b466ae1f4cd9a20f176b800520cc/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:25:44.830207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 196/391] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:25:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:44.839143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c351e-31c3-4913-9151-f530e04f9808 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:45.004845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:45.012699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c351e-31c3-4913-9151-f530e04f9808 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:45.062967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6374b-6119-49be-9f9f-8d4cf9d4a981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:45.063640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6374b-6119-49be-9f9f-8d4cf9d4a981 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:45.063917 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6374b-6119-49be-9f9f-8d4cf9d4a981 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:45.064186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6374b-6119-49be-9f9f-8d4cf9d4a981 None admin] PATH_INFO: `/v3/groups/33a5b466ae1f4cd9a20f176b800520cc` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:45.066037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6374b-6119-49be-9f9f-8d4cf9d4a981 None admin] RBAC: Authorizing `identity:get_group(group_id=33a5b466ae1f4cd9a20f176b800520cc)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:45.067699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6374b-6119-49be-9f9f-8d4cf9d4a981 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:45.069451 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 196/392] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:44 2022] GET /identity/v3/groups/33a5b466ae1f4cd9a20f176b800520cc => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:45.078868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997fefb-e794-4b14-9269-dee58ae080b5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:45.116382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:45.227405 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997fefb-e794-4b14-9269-dee58ae080b5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:45.305995 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634c927-fdb3-489a-86f1-c14cd5f68c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:45.307091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634c927-fdb3-489a-86f1-c14cd5f68c7d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:45.307585 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634c927-fdb3-489a-86f1-c14cd5f68c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:45.307895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634c927-fdb3-489a-86f1-c14cd5f68c7d None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:45.309627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634c927-fdb3-489a-86f1-c14cd5f68c7d None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:45.312484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634c927-fdb3-489a-86f1-c14cd5f68c7d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:45.316383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 197/393] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:45 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:45.327272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6374b-6119-49be-9f9f-8d4cf9d4a981 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:45.483272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6374b-6119-49be-9f9f-8d4cf9d4a981 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:45.532385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ee94f-2f1f-4245-8c50-c6bbb3c5bd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:45.533071 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ee94f-2f1f-4245-8c50-c6bbb3c5bd73 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:45.533399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ee94f-2f1f-4245-8c50-c6bbb3c5bd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:45.533674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ee94f-2f1f-4245-8c50-c6bbb3c5bd73 None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7/groups/33a5b466ae1f4cd9a20f176b800520cc/roles/65f33fe3045c433b9a3e045e7572aed6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:45.536696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ee94f-2f1f-4245-8c50-c6bbb3c5bd73 None admin] RBAC: Authorizing `identity:create_grant(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7, group_id=33a5b466ae1f4cd9a20f176b800520cc, role_id=65f33fe3045c433b9a3e045e7572aed6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:45.538253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ee94f-2f1f-4245-8c50-c6bbb3c5bd73 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:45.629463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 197/394] 10.222.0.44 () {58 vars in 1442 bytes} [Sun Sep 11 08:25:45 2022] PUT /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7/groups/33a5b466ae1f4cd9a20f176b800520cc/roles/65f33fe3045c433b9a3e045e7572aed6 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:25:46.015242 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17dcb6-2b85-4fc6-8d1d-9f6f9c16ea8c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:46.015738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17dcb6-2b85-4fc6-8d1d-9f6f9c16ea8c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:46.016090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17dcb6-2b85-4fc6-8d1d-9f6f9c16ea8c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:46.019099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 198/395] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:46.025035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a852e-53db-40cc-a090-03335bd05b6f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:46.025987 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a852e-53db-40cc-a090-03335bd05b6f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:46.026480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a852e-53db-40cc-a090-03335bd05b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:46.052052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ec4a852e-53db-40cc-a090-03335bd05b6f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:46.053622 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ec4a852e-53db-40cc-a090-03335bd05b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:46.073261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:46.145051 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 198/396] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:46.433981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257757-fa72-4e7a-8bd5-9ab22ffd1a9e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:46.434761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257757-fa72-4e7a-8bd5-9ab22ffd1a9e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:46.435306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257757-fa72-4e7a-8bd5-9ab22ffd1a9e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:46.437205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 199/397] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:46.443188 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fead026-285f-414e-8ecd-f8b7fe4061a9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:46.443819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fead026-285f-414e-8ecd-f8b7fe4061a9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:46.444288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fead026-285f-414e-8ecd-f8b7fe4061a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:46.474495 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6fead026-285f-414e-8ecd-f8b7fe4061a9 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:46.476140 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6fead026-285f-414e-8ecd-f8b7fe4061a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:46.582832 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8257757-fa72-4e7a-8bd5-9ab22ffd1a9e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:46.587706 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 199/398] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:46.713382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:46.735462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8257757-fa72-4e7a-8bd5-9ab22ffd1a9e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:46.787502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303bd27-ea5e-4899-95f9-49fcbcf041f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:46.788143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303bd27-ea5e-4899-95f9-49fcbcf041f8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:46.788418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303bd27-ea5e-4899-95f9-49fcbcf041f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:46.788686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303bd27-ea5e-4899-95f9-49fcbcf041f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:46.803280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0303bd27-ea5e-4899-95f9-49fcbcf041f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0303bd27-ea5e-4899-95f9-49fcbcf041f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:46.808316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:46.809214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:46.810624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:46.810624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:46.810624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:46.810624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:46.810624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:46.810624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:46.810624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:46.810624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:46.810624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:46.810624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:46.810624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:46.810624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:46.810624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 200/399] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:25:46.815982 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fead026-285f-414e-8ecd-f8b7fe4061a9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:46.976438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:46.982103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303bd27-ea5e-4899-95f9-49fcbcf041f8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:47.007035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fead026-285f-414e-8ecd-f8b7fe4061a9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:47.101455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14678dc2-b1d8-4d18-99f6-98b2324c800b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:47.102460 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14678dc2-b1d8-4d18-99f6-98b2324c800b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:47.102939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14678dc2-b1d8-4d18-99f6-98b2324c800b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:47.103413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14678dc2-b1d8-4d18-99f6-98b2324c800b None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:47.104710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14678dc2-b1d8-4d18-99f6-98b2324c800b None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:47.105255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14678dc2-b1d8-4d18-99f6-98b2324c800b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:47.107761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14678dc2-b1d8-4d18-99f6-98b2324c800b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:47.141595 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 200/400] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:47.145899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14678dc2-b1d8-4d18-99f6-98b2324c800b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:47.300677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303bd27-ea5e-4899-95f9-49fcbcf041f8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:47.342985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82375c47-396a-4bb1-ae3d-6b8a525f237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:47.343870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82375c47-396a-4bb1-ae3d-6b8a525f237d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:47.344244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82375c47-396a-4bb1-ae3d-6b8a525f237d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:47.344633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82375c47-396a-4bb1-ae3d-6b8a525f237d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:47.360534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82375c47-396a-4bb1-ae3d-6b8a525f237d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:47.365728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14678dc2-b1d8-4d18-99f6-98b2324c800b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-82375c47-396a-4bb1-ae3d-6b8a525f237d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:47.367529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:47.368978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:47.370766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:47.370766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:47.370766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:47.370766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:47.370766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:47.370766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:47.370766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:47.370766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:47.370766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:47.370766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:47.370766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:47.370766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:47.374257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 201/401] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:25:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:25:47.377607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d931df-86f5-42cc-8541-a78c2c8bd7e8 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:47.377903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d931df-86f5-42cc-8541-a78c2c8bd7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:47.378210 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d931df-86f5-42cc-8541-a78c2c8bd7e8 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:47.384578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 202/402] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:47.384578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d931df-86f5-42cc-8541-a78c2c8bd7e8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:47.388413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:47.421837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10e531-7e25-4a3e-b417-9b3381e1ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:47.423168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10e531-7e25-4a3e-b417-9b3381e1ee40 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:47.424210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10e531-7e25-4a3e-b417-9b3381e1ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:47.424735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10e531-7e25-4a3e-b417-9b3381e1ee40 None admin] PATH_INFO: `/v3/groups/33a5b466ae1f4cd9a20f176b800520cc` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:47.426498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10e531-7e25-4a3e-b417-9b3381e1ee40 None admin] RBAC: Authorizing `identity:get_group(group_id=33a5b466ae1f4cd9a20f176b800520cc)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:47.428212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10e531-7e25-4a3e-b417-9b3381e1ee40 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:47.431162 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 201/403] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:47 2022] GET /identity/v3/groups/33a5b466ae1f4cd9a20f176b800520cc => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:47.432936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71cc6d-5cf5-4b46-b75e-443e4a8c9308 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:47.433429 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71cc6d-5cf5-4b46-b75e-443e4a8c9308 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:47.433939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71cc6d-5cf5-4b46-b75e-443e4a8c9308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:47.459688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2b71cc6d-5cf5-4b46-b75e-443e4a8c9308 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:47.461279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2b71cc6d-5cf5-4b46-b75e-443e4a8c9308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:47.470530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:47.550929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 202/404] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:47.552313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71cc6d-5cf5-4b46-b75e-443e4a8c9308 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:47.612910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d931df-86f5-42cc-8541-a78c2c8bd7e8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:47.734583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47259f-72c9-486a-97a2-934bc492d90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:47.735654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47259f-72c9-486a-97a2-934bc492d90c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:47.736114 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47259f-72c9-486a-97a2-934bc492d90c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:47.736643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47259f-72c9-486a-97a2-934bc492d90c None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:47.737865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47259f-72c9-486a-97a2-934bc492d90c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:47.738428 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47259f-72c9-486a-97a2-934bc492d90c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:47.741064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47259f-72c9-486a-97a2-934bc492d90c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:47.759626 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 203/405] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:25:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:47.771274 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47259f-72c9-486a-97a2-934bc492d90c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:47.832705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71cc6d-5cf5-4b46-b75e-443e4a8c9308 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:47.925276 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4d5a2-7bc2-4b40-a145-38f8ba47fd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:47.925936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4d5a2-7bc2-4b40-a145-38f8ba47fd98 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:47.926260 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4d5a2-7bc2-4b40-a145-38f8ba47fd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:47.926537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4d5a2-7bc2-4b40-a145-38f8ba47fd98 None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:47.927827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4d5a2-7bc2-4b40-a145-38f8ba47fd98 None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:47.929714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4d5a2-7bc2-4b40-a145-38f8ba47fd98 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:47.932841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 203/406] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:47 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:47.933748 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4d5a2-7bc2-4b40-a145-38f8ba47fd98 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:47.943997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:47.951080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:48.000296 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47259f-72c9-486a-97a2-934bc492d90c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:48.041944 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d5e1a-1ea4-43ca-81ab-f6e271a7b0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:48.042622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d5e1a-1ea4-43ca-81ab-f6e271a7b0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:48.042901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d5e1a-1ea4-43ca-81ab-f6e271a7b0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:48.043174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d5e1a-1ea4-43ca-81ab-f6e271a7b0f9 None admin] PATH_INFO: `/v3/groups/33a5b466ae1f4cd9a20f176b800520cc` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:48.044803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d5e1a-1ea4-43ca-81ab-f6e271a7b0f9 None admin] RBAC: Authorizing `identity:get_group(group_id=33a5b466ae1f4cd9a20f176b800520cc)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:48.046364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d5e1a-1ea4-43ca-81ab-f6e271a7b0f9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:48.048122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 204/407] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:47 2022] GET /identity/v3/groups/33a5b466ae1f4cd9a20f176b800520cc => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:48.048915 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d5e1a-1ea4-43ca-81ab-f6e271a7b0f9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:48.121552 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4d5a2-7bc2-4b40-a145-38f8ba47fd98 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:48.165280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d5e1a-1ea4-43ca-81ab-f6e271a7b0f9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:48.187819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6c49d-9bca-42ff-88de-145f819a5c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:48.188854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6c49d-9bca-42ff-88de-145f819a5c7c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:48.189327 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6c49d-9bca-42ff-88de-145f819a5c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:48.189803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6c49d-9bca-42ff-88de-145f819a5c7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:48.206262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970039b2-b46f-4b56-acf8-ea84de6e2773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:48.207001 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970039b2-b46f-4b56-acf8-ea84de6e2773 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:48.207280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970039b2-b46f-4b56-acf8-ea84de6e2773 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:48.207616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970039b2-b46f-4b56-acf8-ea84de6e2773 None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920/groups/33a5b466ae1f4cd9a20f176b800520cc/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:48.209755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970039b2-b46f-4b56-acf8-ea84de6e2773 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac74b750ef33436a83f5ee0474a71920, group_id=33a5b466ae1f4cd9a20f176b800520cc, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:48.211283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970039b2-b46f-4b56-acf8-ea84de6e2773 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:48.212453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6c49d-9bca-42ff-88de-145f819a5c7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:48.213748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-73f6c49d-9bca-42ff-88de-145f819a5c7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:48.223069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:48.231001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 204/408] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:25:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:48.231794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6c49d-9bca-42ff-88de-145f819a5c7c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:48.336971 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 205/409] 10.222.0.44 () {58 vars in 1442 bytes} [Sun Sep 11 08:25:48 2022] PUT /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920/groups/33a5b466ae1f4cd9a20f176b800520cc/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:25:48.339273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970039b2-b46f-4b56-acf8-ea84de6e2773 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:48.364921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:48.429079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6c49d-9bca-42ff-88de-145f819a5c7c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:48.491138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970039b2-b46f-4b56-acf8-ea84de6e2773 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:48.507818 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3333ac-709e-4b16-984d-bf4e4a1c437f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:48.508907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3333ac-709e-4b16-984d-bf4e4a1c437f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:48.509334 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3333ac-709e-4b16-984d-bf4e4a1c437f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:48.509717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3333ac-709e-4b16-984d-bf4e4a1c437f None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:48.512086 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3333ac-709e-4b16-984d-bf4e4a1c437f None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:48.515718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3333ac-709e-4b16-984d-bf4e4a1c437f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:48.520303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 205/410] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:48 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:48.523115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbe9c2-2ae1-4914-a089-5d5fee196dcb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:48.523671 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbe9c2-2ae1-4914-a089-5d5fee196dcb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:48.524143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbe9c2-2ae1-4914-a089-5d5fee196dcb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:48.526891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 206/411] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:48.533758 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889c32-fad7-4cb8-9403-e3fbae67e69d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:48.534054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889c32-fad7-4cb8-9403-e3fbae67e69d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:48.534364 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889c32-fad7-4cb8-9403-e3fbae67e69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:48.542812 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0723e9-9e75-4e42-a8e0-3c154ab0b400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:48.547130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0723e9-9e75-4e42-a8e0-3c154ab0b400 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:48.547130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0723e9-9e75-4e42-a8e0-3c154ab0b400 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:48.547130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0723e9-9e75-4e42-a8e0-3c154ab0b400 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:48.547130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0723e9-9e75-4e42-a8e0-3c154ab0b400 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:48.547130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0723e9-9e75-4e42-a8e0-3c154ab0b400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:48.547130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0723e9-9e75-4e42-a8e0-3c154ab0b400 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:48.557137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0e889c32-fad7-4cb8-9403-e3fbae67e69d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:48.558455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 206/412] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:48.558971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0e889c32-fad7-4cb8-9403-e3fbae67e69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:48.564309 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0723e9-9e75-4e42-a8e0-3c154ab0b400 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:48.651364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:48.705501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 207/413] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:48.707278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e889c32-fad7-4cb8-9403-e3fbae67e69d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:48.726410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0723e9-9e75-4e42-a8e0-3c154ab0b400 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:48.804317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df85c5-5379-41bb-b7f0-53b049207832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:48.805473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df85c5-5379-41bb-b7f0-53b049207832 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:48.805885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df85c5-5379-41bb-b7f0-53b049207832 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:48.806357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df85c5-5379-41bb-b7f0-53b049207832 None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920/groups/33a5b466ae1f4cd9a20f176b800520cc/roles/65f33fe3045c433b9a3e045e7572aed6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:48.811731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df85c5-5379-41bb-b7f0-53b049207832 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac74b750ef33436a83f5ee0474a71920, group_id=33a5b466ae1f4cd9a20f176b800520cc, role_id=65f33fe3045c433b9a3e045e7572aed6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:48.820241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df85c5-5379-41bb-b7f0-53b049207832 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:48.870837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:48.913297 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e889c32-fad7-4cb8-9403-e3fbae67e69d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:48.932315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 207/414] 10.222.0.44 () {58 vars in 1442 bytes} [Sun Sep 11 08:25:48 2022] PUT /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920/groups/33a5b466ae1f4cd9a20f176b800520cc/roles/65f33fe3045c433b9a3e045e7572aed6 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:25:48.935875 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5811af-2299-46d6-b8e8-ade97bb99809 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:48.936775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5811af-2299-46d6-b8e8-ade97bb99809 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:48.937512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5811af-2299-46d6-b8e8-ade97bb99809 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:48.942532 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 208/415] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:48.949029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4be01f-2412-4a4f-b88b-97f90ccb8850 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:48.950517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4be01f-2412-4a4f-b88b-97f90ccb8850 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:48.951697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4be01f-2412-4a4f-b88b-97f90ccb8850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:48.963319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d4896-ef62-43f2-8b1c-832ac4ab1a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:48.963987 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d4896-ef62-43f2-8b1c-832ac4ab1a85 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:48.964360 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d4896-ef62-43f2-8b1c-832ac4ab1a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:48.970185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d4896-ef62-43f2-8b1c-832ac4ab1a85 None admin] PATH_INFO: `/v3/groups/8d6fc93d97044403ac1b6ea456cac2bf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:48.970185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d4896-ef62-43f2-8b1c-832ac4ab1a85 None admin] RBAC: Authorizing `identity:get_group(group_id=8d6fc93d97044403ac1b6ea456cac2bf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:48.970185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d4896-ef62-43f2-8b1c-832ac4ab1a85 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:48.970185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 208/416] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:48 2022] GET /identity/v3/groups/8d6fc93d97044403ac1b6ea456cac2bf => generated 229 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:48.975217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d4896-ef62-43f2-8b1c-832ac4ab1a85 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:49.004247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ed4be01f-2412-4a4f-b88b-97f90ccb8850 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:49.005475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ed4be01f-2412-4a4f-b88b-97f90ccb8850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:49.158314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d4896-ef62-43f2-8b1c-832ac4ab1a85 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:49.172287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 209/417] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:49.174413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4be01f-2412-4a4f-b88b-97f90ccb8850 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:49.211838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f5fab-e1c5-4bfa-8f49-007217f7ba7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:49.212659 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f5fab-e1c5-4bfa-8f49-007217f7ba7f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:49.213266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f5fab-e1c5-4bfa-8f49-007217f7ba7f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:49.213826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f5fab-e1c5-4bfa-8f49-007217f7ba7f None admin] PATH_INFO: `/v3/projects/878028e7aaff4c28815b9cd862f1c36e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:49.216401 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f5fab-e1c5-4bfa-8f49-007217f7ba7f None admin] RBAC: Authorizing `identity:get_project(project_id=878028e7aaff4c28815b9cd862f1c36e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:49.220063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f5fab-e1c5-4bfa-8f49-007217f7ba7f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:49.224483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 209/418] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:48 2022] GET /identity/v3/projects/878028e7aaff4c28815b9cd862f1c36e => generated 344 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:49.231383 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f5fab-e1c5-4bfa-8f49-007217f7ba7f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:49.248443 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:49.283080 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:49.432306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4be01f-2412-4a4f-b88b-97f90ccb8850 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:49.436937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f5fab-e1c5-4bfa-8f49-007217f7ba7f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:49.497421 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f98903-f0d1-41b9-a658-cedf4ce8876a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:49.497884 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f98903-f0d1-41b9-a658-cedf4ce8876a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:49.498445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f98903-f0d1-41b9-a658-cedf4ce8876a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:49.498857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f98903-f0d1-41b9-a658-cedf4ce8876a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:49.512127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78687b-ea80-4d24-b45c-93526fb30053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:49.513989 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78687b-ea80-4d24-b45c-93526fb30053 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:49.514198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f98903-f0d1-41b9-a658-cedf4ce8876a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:49.514876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78687b-ea80-4d24-b45c-93526fb30053 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:49.515548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78687b-ea80-4d24-b45c-93526fb30053 None admin] PATH_INFO: `/v3/projects/878028e7aaff4c28815b9cd862f1c36e/groups/8d6fc93d97044403ac1b6ea456cac2bf/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-00f98903-f0d1-41b9-a658-cedf4ce8876a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:49.520205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:49.521237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:49.522099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:49.522099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:49.522099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:49.522099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:49.522099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:49.522099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:49.522099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:49.522099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:49.522099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:49.522099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:49.522099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:49.522099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:49.522728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78687b-ea80-4d24-b45c-93526fb30053 None admin] RBAC: Authorizing `identity:create_grant(project_id=878028e7aaff4c28815b9cd862f1c36e, group_id=8d6fc93d97044403ac1b6ea456cac2bf, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:49.523154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78687b-ea80-4d24-b45c-93526fb30053 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:49.524505 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 210/419] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:49.525457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f98903-f0d1-41b9-a658-cedf4ce8876a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:49.658122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 210/420] 10.222.0.44 () {58 vars in 1442 bytes} [Sun Sep 11 08:25:49 2022] PUT /identity/v3/projects/878028e7aaff4c28815b9cd862f1c36e/groups/8d6fc93d97044403ac1b6ea456cac2bf/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:25:49.661189 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78687b-ea80-4d24-b45c-93526fb30053 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:49.670059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f98903-f0d1-41b9-a658-cedf4ce8876a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:49.684948 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:49.793286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831ab1f-5ad5-4bce-b980-e3c4155f3a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:49.794557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831ab1f-5ad5-4bce-b980-e3c4155f3a79 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:49.795164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831ab1f-5ad5-4bce-b980-e3c4155f3a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:49.795607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831ab1f-5ad5-4bce-b980-e3c4155f3a79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:49.817970 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:49.844655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831ab1f-5ad5-4bce-b980-e3c4155f3a79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1831ab1f-5ad5-4bce-b980-e3c4155f3a79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:49.859738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:49.861464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:49.863081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:49.863081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:49.863081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:49.863081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:49.863081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:49.863081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:49.863081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:49.863081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:49.863081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:49.863081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:49.863081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:49.863081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:49.868019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 211/421] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:25:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:49.874352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831ab1f-5ad5-4bce-b980-e3c4155f3a79 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:49.908133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78687b-ea80-4d24-b45c-93526fb30053 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:49.967666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d552d79-a159-4c88-8a70-6c6b169a7df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:49.968303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d552d79-a159-4c88-8a70-6c6b169a7df2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:49.968576 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d552d79-a159-4c88-8a70-6c6b169a7df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:49.968846 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d552d79-a159-4c88-8a70-6c6b169a7df2 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:49.969540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d552d79-a159-4c88-8a70-6c6b169a7df2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:49.969841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d552d79-a159-4c88-8a70-6c6b169a7df2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:49.972029 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d552d79-a159-4c88-8a70-6c6b169a7df2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:49.982810 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 211/422] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:49.992914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d552d79-a159-4c88-8a70-6c6b169a7df2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:50.108693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831ab1f-5ad5-4bce-b980-e3c4155f3a79 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:50.162940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d552d79-a159-4c88-8a70-6c6b169a7df2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:50.187272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14601ee6-a0f5-4dbc-b822-6baecb0c4d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:50.187894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14601ee6-a0f5-4dbc-b822-6baecb0c4d19 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:50.188175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14601ee6-a0f5-4dbc-b822-6baecb0c4d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:50.188468 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14601ee6-a0f5-4dbc-b822-6baecb0c4d19 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:50.189122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14601ee6-a0f5-4dbc-b822-6baecb0c4d19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:50.189427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14601ee6-a0f5-4dbc-b822-6baecb0c4d19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:50.190929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14601ee6-a0f5-4dbc-b822-6baecb0c4d19 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:50.210528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 212/423] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:25:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:50.220364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14601ee6-a0f5-4dbc-b822-6baecb0c4d19 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:50.228836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:50.259109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32214af-3692-41a4-a9aa-580f82864059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:50.260080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32214af-3692-41a4-a9aa-580f82864059 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:50.260544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32214af-3692-41a4-a9aa-580f82864059 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:50.260999 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32214af-3692-41a4-a9aa-580f82864059 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:50.278108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:50.484393 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32214af-3692-41a4-a9aa-580f82864059 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:50.487822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32214af-3692-41a4-a9aa-580f82864059 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:50.504426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14601ee6-a0f5-4dbc-b822-6baecb0c4d19 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:50.611648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f55af3-4ff5-4219-b3f5-b574f5e4d30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:50.615107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f55af3-4ff5-4219-b3f5-b574f5e4d30b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:50.616223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f55af3-4ff5-4219-b3f5-b574f5e4d30b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:50.616961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f55af3-4ff5-4219-b3f5-b574f5e4d30b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:50.820537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f55af3-4ff5-4219-b3f5-b574f5e4d30b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:50.823453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f55af3-4ff5-4219-b3f5-b574f5e4d30b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:50.831623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 212/424] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 839 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:50.840944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32214af-3692-41a4-a9aa-580f82864059 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:50.898711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:50.906297 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:51.037025 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32214af-3692-41a4-a9aa-580f82864059 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:51.081273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 213/425] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 860 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:51.089449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f55af3-4ff5-4219-b3f5-b574f5e4d30b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:51.118545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606c741-a4ff-4d47-a35c-61dbca93434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:51.119207 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606c741-a4ff-4d47-a35c-61dbca93434a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:51.119483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606c741-a4ff-4d47-a35c-61dbca93434a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:51.119759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606c741-a4ff-4d47-a35c-61dbca93434a None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:51.121184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606c741-a4ff-4d47-a35c-61dbca93434a None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:51.122679 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606c741-a4ff-4d47-a35c-61dbca93434a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:51.124612 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 213/426] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:50 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:51.134488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606c741-a4ff-4d47-a35c-61dbca93434a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:51.321422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:51.336298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606c741-a4ff-4d47-a35c-61dbca93434a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:51.351803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f55af3-4ff5-4219-b3f5-b574f5e4d30b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:51.406586 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:51.420786 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8cc7b-1a9e-4ff3-a27c-e9bbc21a2e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:51.421459 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8cc7b-1a9e-4ff3-a27c-e9bbc21a2e9b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:51.421743 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8cc7b-1a9e-4ff3-a27c-e9bbc21a2e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:51.422002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8cc7b-1a9e-4ff3-a27c-e9bbc21a2e9b None admin] PATH_INFO: `/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:51.423381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30766d5-7cd9-42ef-9021-2550950664a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:51.424048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30766d5-7cd9-42ef-9021-2550950664a1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:51.424360 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30766d5-7cd9-42ef-9021-2550950664a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:51.424642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30766d5-7cd9-42ef-9021-2550950664a1 None admin] PATH_INFO: `/v3/groups/33a5b466ae1f4cd9a20f176b800520cc` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:51.425653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8cc7b-1a9e-4ff3-a27c-e9bbc21a2e9b None admin] RBAC: Authorizing `identity:get_project(project_id=472a5dbe7a5e416b857aeeb67d2ea4f7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:51.427237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8cc7b-1a9e-4ff3-a27c-e9bbc21a2e9b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:51.429089 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 214/427] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:51 2022] GET /identity/v3/projects/472a5dbe7a5e416b857aeeb67d2ea4f7 => generated 298 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:51.430494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30766d5-7cd9-42ef-9021-2550950664a1 None admin] RBAC: Authorizing `identity:get_group(group_id=33a5b466ae1f4cd9a20f176b800520cc)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:51.432063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30766d5-7cd9-42ef-9021-2550950664a1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:51.434281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 214/428] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:51 2022] GET /identity/v3/groups/33a5b466ae1f4cd9a20f176b800520cc => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:51.435561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8cc7b-1a9e-4ff3-a27c-e9bbc21a2e9b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:51.446210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30766d5-7cd9-42ef-9021-2550950664a1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:51.664224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30766d5-7cd9-42ef-9021-2550950664a1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:51.700246 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8cc7b-1a9e-4ff3-a27c-e9bbc21a2e9b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:51.742383 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e327e-d0e8-404c-9555-6d61bcd26108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:51.743397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e327e-d0e8-404c-9555-6d61bcd26108 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:51.743886 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e327e-d0e8-404c-9555-6d61bcd26108 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:51.744362 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e327e-d0e8-404c-9555-6d61bcd26108 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:51.745744 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e327e-d0e8-404c-9555-6d61bcd26108 None admin] RBAC: Adding query filter params (group.id=33a5b466ae1f4cd9a20f176b800520cc, role.id=dba75ce7d709483fbbb0027d29e99725, scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:51.746299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e327e-d0e8-404c-9555-6d61bcd26108 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:51.749094 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e327e-d0e8-404c-9555-6d61bcd26108 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:51.776054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 215/429] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 08:25:51 2022] GET /identity/v3/role_assignments?group.id=33a5b466ae1f4cd9a20f176b800520cc&scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7&role.id=dba75ce7d709483fbbb0027d29e99725 => generated 611 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:51.807997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04de7b-776e-4329-a1f4-50648059b1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:51.809192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04de7b-776e-4329-a1f4-50648059b1c2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:51.809683 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04de7b-776e-4329-a1f4-50648059b1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:51.810184 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04de7b-776e-4329-a1f4-50648059b1c2 None admin] PATH_INFO: `/v3/groups/33a5b466ae1f4cd9a20f176b800520cc` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:51.815628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04de7b-776e-4329-a1f4-50648059b1c2 None admin] RBAC: Authorizing `identity:get_group(group_id=33a5b466ae1f4cd9a20f176b800520cc)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:51.817491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04de7b-776e-4329-a1f4-50648059b1c2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:51.819979 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 215/430] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:51 2022] GET /identity/v3/groups/33a5b466ae1f4cd9a20f176b800520cc => generated 226 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:25:51.829080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e327e-d0e8-404c-9555-6d61bcd26108 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:51.934836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:52.117170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e327e-d0e8-404c-9555-6d61bcd26108 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:52.195694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa468c6-fee7-4ea8-9b32-265f4245a262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:52.196638 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa468c6-fee7-4ea8-9b32-265f4245a262 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:52.197108 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa468c6-fee7-4ea8-9b32-265f4245a262 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:52.197579 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa468c6-fee7-4ea8-9b32-265f4245a262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:52.198899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa468c6-fee7-4ea8-9b32-265f4245a262 None admin] RBAC: Adding query filter params (group.id=33a5b466ae1f4cd9a20f176b800520cc, role.id=65f33fe3045c433b9a3e045e7572aed6, scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:52.199445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa468c6-fee7-4ea8-9b32-265f4245a262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:52.202130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa468c6-fee7-4ea8-9b32-265f4245a262 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:52.213920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 216/431] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 08:25:51 2022] GET /identity/v3/role_assignments?group.id=33a5b466ae1f4cd9a20f176b800520cc&scope.project.id=472a5dbe7a5e416b857aeeb67d2ea4f7&role.id=65f33fe3045c433b9a3e045e7572aed6 => generated 611 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:52.508682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54933e4-9d47-4543-b21e-a6b4d04b4684 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:52.508682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54933e4-9d47-4543-b21e-a6b4d04b4684 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:52.508682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54933e4-9d47-4543-b21e-a6b4d04b4684 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:52.508682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 216/432] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:52.514074 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45534e-2f35-4272-939d-268f10143d25 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:52.514875 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45534e-2f35-4272-939d-268f10143d25 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:52.515037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45534e-2f35-4272-939d-268f10143d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:52.542318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1e45534e-2f35-4272-939d-268f10143d25 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:52.543528 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1e45534e-2f35-4272-939d-268f10143d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:52.612898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453eb00-bc7e-495f-9964-1ac9331f5933 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:52.613151 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453eb00-bc7e-495f-9964-1ac9331f5933 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:52.613492 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453eb00-bc7e-495f-9964-1ac9331f5933 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:52.615146 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 217/433] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:52.624275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d93821-5b9e-458d-86ee-0b00853edcd3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:52.624687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d93821-5b9e-458d-86ee-0b00853edcd3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:52.624837 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d93821-5b9e-458d-86ee-0b00853edcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:52.661208 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 217/434] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:52.662926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b1d93821-5b9e-458d-86ee-0b00853edcd3 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:52.664157 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b1d93821-5b9e-458d-86ee-0b00853edcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:52.743621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:52.753781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 218/435] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:52.882597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f4ba7-17c0-4f67-88d3-78526d6cab43 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:52.882975 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f4ba7-17c0-4f67-88d3-78526d6cab43 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:52.883332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f4ba7-17c0-4f67-88d3-78526d6cab43 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:52.885338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 218/436] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:52.892611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c470-cba7-4f19-ac73-8bad3ef0c2b3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:52.893069 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c470-cba7-4f19-ac73-8bad3ef0c2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:52.893765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c470-cba7-4f19-ac73-8bad3ef0c2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:52.914598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e674c470-cba7-4f19-ac73-8bad3ef0c2b3 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:52.915868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e674c470-cba7-4f19-ac73-8bad3ef0c2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:52.993389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f4ba7-17c0-4f67-88d3-78526d6cab43 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:53.021436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:53.029645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 219/437] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:53.050344 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c470-cba7-4f19-ac73-8bad3ef0c2b3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:53.142174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f4ba7-17c0-4f67-88d3-78526d6cab43 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:53.187039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c175d3-cfb6-46dd-ba1c-f7bd8fd235d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:53.187299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c470-cba7-4f19-ac73-8bad3ef0c2b3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:53.187676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c175d3-cfb6-46dd-ba1c-f7bd8fd235d2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:53.187948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c175d3-cfb6-46dd-ba1c-f7bd8fd235d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:53.188216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c175d3-cfb6-46dd-ba1c-f7bd8fd235d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:53.201261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c175d3-cfb6-46dd-ba1c-f7bd8fd235d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f5c175d3-cfb6-46dd-ba1c-f7bd8fd235d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:53.206863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:53.208137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:53.209603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:53.209603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:53.209603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:53.209603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:53.209603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:53.209603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:53.209603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:53.209603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:53.209603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:53.209603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:53.209603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 08:25:53.209603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:53.209603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 219/438] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:53.215956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c175d3-cfb6-46dd-ba1c-f7bd8fd235d2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:53.220817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:53.235488 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c91b1-9206-4009-aa3d-e51e4b267a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:53.236130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c91b1-9206-4009-aa3d-e51e4b267a66 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:53.236417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c91b1-9206-4009-aa3d-e51e4b267a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:53.236723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c91b1-9206-4009-aa3d-e51e4b267a66 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:53.249660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c91b1-9206-4009-aa3d-e51e4b267a66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1b5c91b1-9206-4009-aa3d-e51e4b267a66 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:53.254943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:53.256216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:53.257079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:53.257079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:53.257079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:53.257079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:53.257079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:53.257079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:53.257079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:53.257079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:53.257079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:53.257079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:53.257079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:25:53.257079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:25:53.257079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 220/439] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:25:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:25:53.263117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c91b1-9206-4009-aa3d-e51e4b267a66 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:53.324276 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:53.350443 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c175d3-cfb6-46dd-ba1c-f7bd8fd235d2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:53.387940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7452928-b5ca-4577-82c6-95b4b6ee05a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:53.388553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7452928-b5ca-4577-82c6-95b4b6ee05a6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:53.388830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7452928-b5ca-4577-82c6-95b4b6ee05a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:53.389097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7452928-b5ca-4577-82c6-95b4b6ee05a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:53.389765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7452928-b5ca-4577-82c6-95b4b6ee05a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:53.390062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7452928-b5ca-4577-82c6-95b4b6ee05a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:53.390593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c91b1-9206-4009-aa3d-e51e4b267a66 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:53.391550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7452928-b5ca-4577-82c6-95b4b6ee05a6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:53.402692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 220/440] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:25:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:53.403449 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7452928-b5ca-4577-82c6-95b4b6ee05a6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:53.438120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3150e9-a04a-4b3f-a534-c37b80c5e2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:53.438752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3150e9-a04a-4b3f-a534-c37b80c5e2ea None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:53.439027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3150e9-a04a-4b3f-a534-c37b80c5e2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:53.439297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3150e9-a04a-4b3f-a534-c37b80c5e2ea None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:53.440014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3150e9-a04a-4b3f-a534-c37b80c5e2ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:53.440316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3150e9-a04a-4b3f-a534-c37b80c5e2ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:53.441724 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3150e9-a04a-4b3f-a534-c37b80c5e2ea None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:53.450937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 221/441] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:25:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:53.450937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3150e9-a04a-4b3f-a534-c37b80c5e2ea None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:53.504281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:53.527445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7452928-b5ca-4577-82c6-95b4b6ee05a6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:53.565576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3150e9-a04a-4b3f-a534-c37b80c5e2ea None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:53.574692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed042d7-c5b9-41ec-a9d8-cdae0c85840c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:53.575326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed042d7-c5b9-41ec-a9d8-cdae0c85840c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:53.575606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed042d7-c5b9-41ec-a9d8-cdae0c85840c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:53.575870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed042d7-c5b9-41ec-a9d8-cdae0c85840c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:53.588186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed042d7-c5b9-41ec-a9d8-cdae0c85840c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8ed042d7-c5b9-41ec-a9d8-cdae0c85840c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:25:53.594003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:53.598840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 221/442] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:25:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:53.599639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed042d7-c5b9-41ec-a9d8-cdae0c85840c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:53.601080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:53.607738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2fdca-ad1d-402c-9d7e-9626f56cfdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:53.623402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2fdca-ad1d-402c-9d7e-9626f56cfdd7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:53.623402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2fdca-ad1d-402c-9d7e-9626f56cfdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:53.623402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2fdca-ad1d-402c-9d7e-9626f56cfdd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:53.689346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2fdca-ad1d-402c-9d7e-9626f56cfdd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:53.691642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2fdca-ad1d-402c-9d7e-9626f56cfdd7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:53.745263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed042d7-c5b9-41ec-a9d8-cdae0c85840c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:53.752096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:53.789880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18628e6-9e96-4bb3-9884-939412d20a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:53.790956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18628e6-9e96-4bb3-9884-939412d20a9b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:53.791441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18628e6-9e96-4bb3-9884-939412d20a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:53.791934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18628e6-9e96-4bb3-9884-939412d20a9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:53.819054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 222/443] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:53.819937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2fdca-ad1d-402c-9d7e-9626f56cfdd7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:53.883466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18628e6-9e96-4bb3-9884-939412d20a9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:53.885258 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18628e6-9e96-4bb3-9884-939412d20a9b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:53.887983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:53.978246 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2fdca-ad1d-402c-9d7e-9626f56cfdd7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:53.997720 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 222/444] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:53.998613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18628e6-9e96-4bb3-9884-939412d20a9b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:54.031359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364679a-4089-46fb-b910-091fad2addba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:54.031975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364679a-4089-46fb-b910-091fad2addba None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:54.032251 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364679a-4089-46fb-b910-091fad2addba None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:54.032515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364679a-4089-46fb-b910-091fad2addba None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:54.033198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364679a-4089-46fb-b910-091fad2addba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:54.033507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364679a-4089-46fb-b910-091fad2addba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:54.035031 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364679a-4089-46fb-b910-091fad2addba None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:54.044985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 223/445] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:54.045862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364679a-4089-46fb-b910-091fad2addba None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:54.059862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:54.141751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18628e6-9e96-4bb3-9884-939412d20a9b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:54.144118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:54.194418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364679a-4089-46fb-b910-091fad2addba None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:54.197450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe9e93-4597-44bc-b464-92e14067f71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:54.198389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe9e93-4597-44bc-b464-92e14067f71f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:54.198714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe9e93-4597-44bc-b464-92e14067f71f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:54.199033 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe9e93-4597-44bc-b464-92e14067f71f None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:54.201040 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe9e93-4597-44bc-b464-92e14067f71f None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:54.203592 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe9e93-4597-44bc-b464-92e14067f71f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:54.211569 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 223/446] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:53 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:54.211569 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe9e93-4597-44bc-b464-92e14067f71f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:54.253004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8ac77-4133-4c87-b46f-dfef84d883fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:54.253835 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8ac77-4133-4c87-b46f-dfef84d883fb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:54.254332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8ac77-4133-4c87-b46f-dfef84d883fb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:54.254771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8ac77-4133-4c87-b46f-dfef84d883fb None admin] PATH_INFO: `/v3/projects/ac74b750ef33436a83f5ee0474a71920` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:54.256704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8ac77-4133-4c87-b46f-dfef84d883fb None admin] RBAC: Authorizing `identity:get_project(project_id=ac74b750ef33436a83f5ee0474a71920)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:54.259117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8ac77-4133-4c87-b46f-dfef84d883fb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:54.262059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 224/447] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:54 2022] GET /identity/v3/projects/ac74b750ef33436a83f5ee0474a71920 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:54.263526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8ac77-4133-4c87-b46f-dfef84d883fb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:54.357436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe9e93-4597-44bc-b464-92e14067f71f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:54.369286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:54.404053 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8ac77-4133-4c87-b46f-dfef84d883fb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:54.413713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17a575-0aea-4224-b1ba-fd19c7d831df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:54.413713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17a575-0aea-4224-b1ba-fd19c7d831df None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:54.414674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17a575-0aea-4224-b1ba-fd19c7d831df None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:54.414674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17a575-0aea-4224-b1ba-fd19c7d831df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:54.442639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aac656-d9b0-4013-bb57-fea38d326737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:54.443273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aac656-d9b0-4013-bb57-fea38d326737 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:54.443551 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aac656-d9b0-4013-bb57-fea38d326737 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:54.443816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aac656-d9b0-4013-bb57-fea38d326737 None admin] PATH_INFO: `/v3/groups/33a5b466ae1f4cd9a20f176b800520cc` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:54.445011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aac656-d9b0-4013-bb57-fea38d326737 None admin] RBAC: Authorizing `identity:get_group(group_id=33a5b466ae1f4cd9a20f176b800520cc)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:54.446496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aac656-d9b0-4013-bb57-fea38d326737 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:54.448116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 225/448] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:54 2022] GET /identity/v3/groups/33a5b466ae1f4cd9a20f176b800520cc => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:54.448898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aac656-d9b0-4013-bb57-fea38d326737 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:54.462952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:54.496042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17a575-0aea-4224-b1ba-fd19c7d831df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:54.497504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17a575-0aea-4224-b1ba-fd19c7d831df None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:54.568008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aac656-d9b0-4013-bb57-fea38d326737 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:54.607869 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 224/449] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:25:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:25:54.608437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861306a9-eef3-4317-95c3-a943f2a12e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:54.608692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861306a9-eef3-4317-95c3-a943f2a12e2d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:54.608844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17a575-0aea-4224-b1ba-fd19c7d831df None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:54.609001 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861306a9-eef3-4317-95c3-a943f2a12e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:54.609244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861306a9-eef3-4317-95c3-a943f2a12e2d None admin] PATH_INFO: `/v3/groups/33a5b466ae1f4cd9a20f176b800520cc` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:54.610451 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861306a9-eef3-4317-95c3-a943f2a12e2d None admin] RBAC: Authorizing `identity:get_group(group_id=33a5b466ae1f4cd9a20f176b800520cc)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:54.612538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861306a9-eef3-4317-95c3-a943f2a12e2d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:54.614249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 226/450] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:54 2022] GET /identity/v3/groups/33a5b466ae1f4cd9a20f176b800520cc => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:54.615012 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861306a9-eef3-4317-95c3-a943f2a12e2d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:54.631534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:54.694171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:54.720363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861306a9-eef3-4317-95c3-a943f2a12e2d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:54.729614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17a575-0aea-4224-b1ba-fd19c7d831df None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:54.755772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5c214-b1e5-4aa9-9a9b-340d0e7ed90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:54.756395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5c214-b1e5-4aa9-9a9b-340d0e7ed90f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:54.756677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5c214-b1e5-4aa9-9a9b-340d0e7ed90f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:54.756944 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5c214-b1e5-4aa9-9a9b-340d0e7ed90f None admin] PATH_INFO: `/v3/projects/878028e7aaff4c28815b9cd862f1c36e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:54.758169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5c214-b1e5-4aa9-9a9b-340d0e7ed90f None admin] RBAC: Authorizing `identity:get_project(project_id=878028e7aaff4c28815b9cd862f1c36e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:54.759620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5c214-b1e5-4aa9-9a9b-340d0e7ed90f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:54.761238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 227/451] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:25:54 2022] GET /identity/v3/projects/878028e7aaff4c28815b9cd862f1c36e => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:54.761999 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5c214-b1e5-4aa9-9a9b-340d0e7ed90f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:54.771854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc6048-652c-48ce-9604-ae73925a38dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:54.772478 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc6048-652c-48ce-9604-ae73925a38dd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:54.772763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc6048-652c-48ce-9604-ae73925a38dd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:54.773036 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc6048-652c-48ce-9604-ae73925a38dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:54.774136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc6048-652c-48ce-9604-ae73925a38dd None admin] RBAC: Adding query filter params (group.id=33a5b466ae1f4cd9a20f176b800520cc, role.id=dba75ce7d709483fbbb0027d29e99725, scope.project.id=ac74b750ef33436a83f5ee0474a71920) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:54.774480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc6048-652c-48ce-9604-ae73925a38dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:54.775928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc6048-652c-48ce-9604-ae73925a38dd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:54.781967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 225/452] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 08:25:54 2022] GET /identity/v3/role_assignments?group.id=33a5b466ae1f4cd9a20f176b800520cc&scope.project.id=ac74b750ef33436a83f5ee0474a71920&role.id=dba75ce7d709483fbbb0027d29e99725 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:54.782822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc6048-652c-48ce-9604-ae73925a38dd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:54.858329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:54.887772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5c214-b1e5-4aa9-9a9b-340d0e7ed90f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:54.926378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc6048-652c-48ce-9604-ae73925a38dd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:54.929752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4452554-9155-4d1c-b325-b9676b5a1978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:54.929947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4452554-9155-4d1c-b325-b9676b5a1978 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:54.930221 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4452554-9155-4d1c-b325-b9676b5a1978 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:54.930496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4452554-9155-4d1c-b325-b9676b5a1978 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:54.931230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4452554-9155-4d1c-b325-b9676b5a1978 None admin] RBAC: Adding query filter params (group.id=33a5b466ae1f4cd9a20f176b800520cc, role.id=65f33fe3045c433b9a3e045e7572aed6, scope.project.id=ac74b750ef33436a83f5ee0474a71920) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:54.931536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4452554-9155-4d1c-b325-b9676b5a1978 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:54.933057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4452554-9155-4d1c-b325-b9676b5a1978 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:54.950179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 228/453] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 08:25:54 2022] GET /identity/v3/role_assignments?group.id=33a5b466ae1f4cd9a20f176b800520cc&scope.project.id=ac74b750ef33436a83f5ee0474a71920&role.id=65f33fe3045c433b9a3e045e7572aed6 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:54.974165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb696964-ec7c-4bd8-8aa5-7a5d044ff0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:54.974852 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb696964-ec7c-4bd8-8aa5-7a5d044ff0da None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:54.975133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb696964-ec7c-4bd8-8aa5-7a5d044ff0da None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:54.975405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb696964-ec7c-4bd8-8aa5-7a5d044ff0da None admin] PATH_INFO: `/v3/groups/8d6fc93d97044403ac1b6ea456cac2bf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:54.976651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb696964-ec7c-4bd8-8aa5-7a5d044ff0da None admin] RBAC: Authorizing `identity:get_group(group_id=8d6fc93d97044403ac1b6ea456cac2bf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:54.978123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb696964-ec7c-4bd8-8aa5-7a5d044ff0da None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:54.980052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 226/454] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 08:25:54 2022] GET /identity/v3/groups/8d6fc93d97044403ac1b6ea456cac2bf => generated 229 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:54.986070 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4452554-9155-4d1c-b325-b9676b5a1978 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:55.113988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4452554-9155-4d1c-b325-b9676b5a1978 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:55.158760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831964e-b41a-453a-8ef6-f7861363cfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:55.159379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831964e-b41a-453a-8ef6-f7861363cfda None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:55.159650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831964e-b41a-453a-8ef6-f7861363cfda None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:55.159917 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831964e-b41a-453a-8ef6-f7861363cfda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:55.160648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831964e-b41a-453a-8ef6-f7861363cfda None admin] RBAC: Adding query filter params (group.id=8d6fc93d97044403ac1b6ea456cac2bf, role.id=020bc02dca71485bab2312442adaa179, scope.project.id=878028e7aaff4c28815b9cd862f1c36e) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:25:55.160949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831964e-b41a-453a-8ef6-f7861363cfda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:55.162439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831964e-b41a-453a-8ef6-f7861363cfda None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:55.170233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 229/455] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 08:25:54 2022] GET /identity/v3/role_assignments?group.id=8d6fc93d97044403ac1b6ea456cac2bf&scope.project.id=878028e7aaff4c28815b9cd862f1c36e&role.id=020bc02dca71485bab2312442adaa179 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:25:58.769894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce17b9-2b43-4bd8-b696-1d34f3bc45d3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:58.770718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce17b9-2b43-4bd8-b696-1d34f3bc45d3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:58.770881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce17b9-2b43-4bd8-b696-1d34f3bc45d3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:58.773608 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 227/456] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:58.787277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56010b6-3a03-4c89-b74f-355cebdfa6c2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:58.788683 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56010b6-3a03-4c89-b74f-355cebdfa6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:58.789450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56010b6-3a03-4c89-b74f-355cebdfa6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:58.830210 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c56010b6-3a03-4c89-b74f-355cebdfa6c2 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:58.833273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c56010b6-3a03-4c89-b74f-355cebdfa6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:58.837272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:58.909346 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b419e-9e62-4745-bb43-ea7196a10aab None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:58.910309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b419e-9e62-4745-bb43-ea7196a10aab None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:58.911106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b419e-9e62-4745-bb43-ea7196a10aab None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:58.918785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 228/457] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:25:58.924877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe80ba-1ecd-4c28-b0aa-fe9746954ac9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:58.925762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe80ba-1ecd-4c28-b0aa-fe9746954ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:58.926543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe80ba-1ecd-4c28-b0aa-fe9746954ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:58.944462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:58.987354 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4afe80ba-1ecd-4c28-b0aa-fe9746954ac9 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:58.988854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4afe80ba-1ecd-4c28-b0aa-fe9746954ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:59.040805 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 230/458] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:59.058260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc17e0-eb94-40b6-84fb-f4697cbd66ec None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:59.059176 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc17e0-eb94-40b6-84fb-f4697cbd66ec None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:59.060123 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc17e0-eb94-40b6-84fb-f4697cbd66ec None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:59.063318 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 231/459] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:25:59.072638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d7375-601f-48e5-a0b0-a06feb0572d3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:59.073645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d7375-601f-48e5-a0b0-a06feb0572d3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:59.074449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d7375-601f-48e5-a0b0-a06feb0572d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:59.107275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-015d7375-601f-48e5-a0b0-a06feb0572d3 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:59.108499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-015d7375-601f-48e5-a0b0-a06feb0572d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:59.138173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 229/460] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:59.232596 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 232/461] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:59.390923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04523e4a-68fb-4eee-87bb-7a11e40c912c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:59.391302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04523e4a-68fb-4eee-87bb-7a11e40c912c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:59.392025 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04523e4a-68fb-4eee-87bb-7a11e40c912c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:59.395752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 230/462] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:25:59.405081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa6bcf-b4f4-43d9-ba44-7ced30b90928 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:59.405625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa6bcf-b4f4-43d9-ba44-7ced30b90928 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:59.406039 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa6bcf-b4f4-43d9-ba44-7ced30b90928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:59.430260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-20fa6bcf-b4f4-43d9-ba44-7ced30b90928 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:59.431696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-20fa6bcf-b4f4-43d9-ba44-7ced30b90928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:59.489994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04523e4a-68fb-4eee-87bb-7a11e40c912c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:59.527583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:59.534736 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 233/463] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:25:59.607612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28118c-2e1c-4880-8929-d27f349ed39f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:59.608228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28118c-2e1c-4880-8929-d27f349ed39f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:59.608684 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28118c-2e1c-4880-8929-d27f349ed39f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:59.610257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 234/464] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:25:59.622475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35999d5-bf69-42e0-acc4-5c173b7e8226 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:59.623017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35999d5-bf69-42e0-acc4-5c173b7e8226 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:59.623446 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35999d5-bf69-42e0-acc4-5c173b7e8226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:59.652784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d35999d5-bf69-42e0-acc4-5c173b7e8226 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:25:59.658912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d35999d5-bf69-42e0-acc4-5c173b7e8226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:25:59.716407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04523e4a-68fb-4eee-87bb-7a11e40c912c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:25:59.771725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 235/465] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:25:59.772831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35999d5-bf69-42e0-acc4-5c173b7e8226 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:59.777602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:25:59.783355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dc93f-3909-437a-a435-4a2984476c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:25:59.784257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dc93f-3909-437a-a435-4a2984476c0b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:25:59.784683 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dc93f-3909-437a-a435-4a2984476c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:25:59.785695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dc93f-3909-437a-a435-4a2984476c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:25:59.797639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dc93f-3909-437a-a435-4a2984476c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:25:59.799845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dc93f-3909-437a-a435-4a2984476c0b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7e8dc93f-3909-437a-a435-4a2984476c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:25:59.831649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:25:59.832880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:25:59.833996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:25:59.833996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:25:59.833996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:25:59.838429 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 231/466] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:25:59.838429 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dc93f-3909-437a-a435-4a2984476c0b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:25:59.975948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dc93f-3909-437a-a435-4a2984476c0b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:00.021364 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1164ebf-fbbf-4115-8a5c-f7c9711a83cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:00.022557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1164ebf-fbbf-4115-8a5c-f7c9711a83cf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:00.023058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1164ebf-fbbf-4115-8a5c-f7c9711a83cf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:00.023500 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1164ebf-fbbf-4115-8a5c-f7c9711a83cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:00.026650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35999d5-bf69-42e0-acc4-5c173b7e8226 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:00.046756 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1164ebf-fbbf-4115-8a5c-f7c9711a83cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:00.048470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1164ebf-fbbf-4115-8a5c-f7c9711a83cf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d1164ebf-fbbf-4115-8a5c-f7c9711a83cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:00.066453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:00.067599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:00.068631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:00.068631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:00.068631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:00.070070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 232/467] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:00.071042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1164ebf-fbbf-4115-8a5c-f7c9711a83cf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:00.094945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:00.107314 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8d5e0-a0ab-423f-a230-f3508b25660f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:00.108354 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8d5e0-a0ab-423f-a230-f3508b25660f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:00.111205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8d5e0-a0ab-423f-a230-f3508b25660f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:00.111205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8d5e0-a0ab-423f-a230-f3508b25660f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:00.112446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:00.126846 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8d5e0-a0ab-423f-a230-f3508b25660f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:00.129104 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8d5e0-a0ab-423f-a230-f3508b25660f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b8c8d5e0-a0ab-423f-a230-f3508b25660f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:00.145833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:00.147117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:00.148169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:00.148169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:00.148169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:00.150539 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 236/468] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:00.152323 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8d5e0-a0ab-423f-a230-f3508b25660f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:00.211198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1164ebf-fbbf-4115-8a5c-f7c9711a83cf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:00.249679 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a298e-5b39-40b5-84ae-bb4dd265d22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:00.250411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a298e-5b39-40b5-84ae-bb4dd265d22e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:00.250692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a298e-5b39-40b5-84ae-bb4dd265d22e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:00.250959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a298e-5b39-40b5-84ae-bb4dd265d22e None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:00.251598 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a298e-5b39-40b5-84ae-bb4dd265d22e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:00.251892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a298e-5b39-40b5-84ae-bb4dd265d22e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:00.253841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a298e-5b39-40b5-84ae-bb4dd265d22e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:00.267132 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 233/469] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:00.268040 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a298e-5b39-40b5-84ae-bb4dd265d22e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:00.306497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8d5e0-a0ab-423f-a230-f3508b25660f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:00.348150 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b0a27-00cb-432d-bcd4-9f47991c8d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:00.348769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b0a27-00cb-432d-bcd4-9f47991c8d7b None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:00.349041 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b0a27-00cb-432d-bcd4-9f47991c8d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:00.349305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b0a27-00cb-432d-bcd4-9f47991c8d7b None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:00.349997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b0a27-00cb-432d-bcd4-9f47991c8d7b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:00.351483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b0a27-00cb-432d-bcd4-9f47991c8d7b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:00.391725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.sql.core [None req-7f8b0a27-00cb-432d-bcd4-9f47991c8d7b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 08:26:00.391725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 11 08:26:00.391725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [parameters: {'id': '5676c2361cab4d4f823b715bdfad96bb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 11 08:26:00.391725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81571) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7f8b0a27-00cb-432d-bcd4-9f47991c8d7b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Sep 11 08:26:00.409443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return ref.to_dict() Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 08:26:00.411325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application yield resource Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise self.value Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application session.commit() Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 08:26:00.413364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.flush() Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application compat.raise_( Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 08:26:00.415912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 08:26:00.417709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 08:26:00.420553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [parameters: {'id': '5676c2361cab4d4f823b715bdfad96bb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:00.422290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 11 08:26:00.424946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:00.427039 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 237/470] 10.222.0.44 () {60 vars in 1131 bytes} [Sun Sep 11 08:26:00 2022] POST /identity/v3/roles => generated 144 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:00.427039 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b0a27-00cb-432d-bcd4-9f47991c8d7b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:00.427272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:00.429673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a298e-5b39-40b5-84ae-bb4dd265d22e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:00.477064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29211be-b542-485b-8f3c-e79a74592684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:00.477764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29211be-b542-485b-8f3c-e79a74592684 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:00.478047 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29211be-b542-485b-8f3c-e79a74592684 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:00.478349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29211be-b542-485b-8f3c-e79a74592684 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:00.479085 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29211be-b542-485b-8f3c-e79a74592684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:00.479388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29211be-b542-485b-8f3c-e79a74592684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:00.480803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29211be-b542-485b-8f3c-e79a74592684 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:00.490105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:00.495029 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 234/471] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:00.495787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29211be-b542-485b-8f3c-e79a74592684 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:00.568239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b0a27-00cb-432d-bcd4-9f47991c8d7b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:00.615425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920d5e9-c09a-42d7-a28c-d312157c5b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:00.616416 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920d5e9-c09a-42d7-a28c-d312157c5b73 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:00.616867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920d5e9-c09a-42d7-a28c-d312157c5b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:00.617306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920d5e9-c09a-42d7-a28c-d312157c5b73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:00.622510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29211be-b542-485b-8f3c-e79a74592684 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:00.635470 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920d5e9-c09a-42d7-a28c-d312157c5b73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:00.638234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920d5e9-c09a-42d7-a28c-d312157c5b73 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:00.658214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b2512-1e42-4267-bb99-6270e90fe20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:00.659365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b2512-1e42-4267-bb99-6270e90fe20f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:00.659639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b2512-1e42-4267-bb99-6270e90fe20f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:00.659910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b2512-1e42-4267-bb99-6270e90fe20f None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:00.660557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b2512-1e42-4267-bb99-6270e90fe20f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:00.660857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b2512-1e42-4267-bb99-6270e90fe20f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:00.662261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b2512-1e42-4267-bb99-6270e90fe20f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f920d5e9-c09a-42d7-a28c-d312157c5b73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:00.664923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:00.665817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:00.666745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:00.666745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:00.666745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:00.666912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 238/472] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:00.669448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920d5e9-c09a-42d7-a28c-d312157c5b73 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:00.682898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 235/473] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:00.682898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b2512-1e42-4267-bb99-6270e90fe20f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:00.697920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:00.790843 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:00.793734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b2512-1e42-4267-bb99-6270e90fe20f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:00.797109 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920d5e9-c09a-42d7-a28c-d312157c5b73 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:00.834113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7714531-ddae-402d-9a87-651e2b0c1c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:00.834672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7714531-ddae-402d-9a87-651e2b0c1c28 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:00.834943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7714531-ddae-402d-9a87-651e2b0c1c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:00.835215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7714531-ddae-402d-9a87-651e2b0c1c28 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:00.835870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7714531-ddae-402d-9a87-651e2b0c1c28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:00.837339 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7714531-ddae-402d-9a87-651e2b0c1c28 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:00.841431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2051a-11d3-4260-bd43-00229bdc9605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:00.842510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2051a-11d3-4260-bd43-00229bdc9605 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:00.842996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2051a-11d3-4260-bd43-00229bdc9605 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:00.843479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2051a-11d3-4260-bd43-00229bdc9605 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:00.860555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2051a-11d3-4260-bd43-00229bdc9605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6dc2051a-11d3-4260-bd43-00229bdc9605 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:00.870723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:00.872596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 236/474] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:26:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:00.879319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2051a-11d3-4260-bd43-00229bdc9605 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:00.947197 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 239/475] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:26:00 2022] POST /identity/v3/users => generated 248 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:26:00.948635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7714531-ddae-402d-9a87-651e2b0c1c28 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:00.990359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:01.035217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2051a-11d3-4260-bd43-00229bdc9605 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:01.080376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e134e-e4cb-42d7-83d9-9270835676c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:01.081007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e134e-e4cb-42d7-83d9-9270835676c0 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:01.081282 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e134e-e4cb-42d7-83d9-9270835676c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:01.081551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e134e-e4cb-42d7-83d9-9270835676c0 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:01.082329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e134e-e4cb-42d7-83d9-9270835676c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:01.083840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e134e-e4cb-42d7-83d9-9270835676c0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:01.116819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7714531-ddae-402d-9a87-651e2b0c1c28 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:01.160677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5472f-6f56-48b9-bc9f-4b0006559979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:01.161310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5472f-6f56-48b9-bc9f-4b0006559979 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:01.161593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5472f-6f56-48b9-bc9f-4b0006559979 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:01.161864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5472f-6f56-48b9-bc9f-4b0006559979 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:01.162816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5472f-6f56-48b9-bc9f-4b0006559979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:01.162962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5472f-6f56-48b9-bc9f-4b0006559979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:01.164267 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5472f-6f56-48b9-bc9f-4b0006559979 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:01.167939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 237/476] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:26:00 2022] POST /identity/v3/users => generated 246 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:26:01.169292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e134e-e4cb-42d7-83d9-9270835676c0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:01.197201 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 240/477] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:01.198506 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5472f-6f56-48b9-bc9f-4b0006559979 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:01.222067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:01.347433 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e134e-e4cb-42d7-83d9-9270835676c0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:01.347798 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5472f-6f56-48b9-bc9f-4b0006559979 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:01.397902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:01.407886 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b030154-306d-417f-9650-0137c595f4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:01.409988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b030154-306d-417f-9650-0137c595f4bf None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:01.409988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b030154-306d-417f-9650-0137c595f4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:01.409988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b030154-306d-417f-9650-0137c595f4bf None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:01.410264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b030154-306d-417f-9650-0137c595f4bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:01.411686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733c65d-2ab0-449e-8728-d2c53e92c746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:01.411840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b030154-306d-417f-9650-0137c595f4bf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:01.412738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733c65d-2ab0-449e-8728-d2c53e92c746 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:01.413225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733c65d-2ab0-449e-8728-d2c53e92c746 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:01.413692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733c65d-2ab0-449e-8728-d2c53e92c746 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:01.415324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733c65d-2ab0-449e-8728-d2c53e92c746 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:01.415469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733c65d-2ab0-449e-8728-d2c53e92c746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:01.418050 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733c65d-2ab0-449e-8728-d2c53e92c746 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:01.433226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 241/478] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:01.434574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733c65d-2ab0-449e-8728-d2c53e92c746 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:01.536570 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 238/479] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:26:01 2022] POST /identity/v3/users => generated 248 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:26:01.561770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:01.606136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733c65d-2ab0-449e-8728-d2c53e92c746 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:01.660332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4e9a3-70cb-4628-8055-fdf2a06ac060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:01.666182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4e9a3-70cb-4628-8055-fdf2a06ac060 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:01.666182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4e9a3-70cb-4628-8055-fdf2a06ac060 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:01.666182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4e9a3-70cb-4628-8055-fdf2a06ac060 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:01.666182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4e9a3-70cb-4628-8055-fdf2a06ac060 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:01.666182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4e9a3-70cb-4628-8055-fdf2a06ac060 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:01.767046 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 242/480] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:26:01 2022] POST /identity/v3/users => generated 249 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:26:04.218314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4921c11-1625-461a-bd5f-341b3617a061 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:04.218314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4921c11-1625-461a-bd5f-341b3617a061 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:04.218314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4921c11-1625-461a-bd5f-341b3617a061 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:04.223317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 239/481] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:04.228979 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aca15f-ef83-4766-b8c6-05c8d8697c07 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:04.230425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aca15f-ef83-4766-b8c6-05c8d8697c07 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:04.231182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aca15f-ef83-4766-b8c6-05c8d8697c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:04.272714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-68aca15f-ef83-4766-b8c6-05c8d8697c07 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:04.275086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-68aca15f-ef83-4766-b8c6-05c8d8697c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:04.312121 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42af1ae-ee5e-4ca7-b688-a40f2f975d2b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:04.312411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42af1ae-ee5e-4ca7-b688-a40f2f975d2b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:04.312680 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42af1ae-ee5e-4ca7-b688-a40f2f975d2b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:04.314399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 240/482] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:04.320265 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c619e9b-6659-4769-8953-62da96291d62 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:04.320647 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c619e9b-6659-4769-8953-62da96291d62 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:04.320819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c619e9b-6659-4769-8953-62da96291d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:04.387558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2c619e9b-6659-4769-8953-62da96291d62 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:04.390671 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c619e9b-6659-4769-8953-62da96291d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:04.454840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 243/483] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:04.508299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 241/484] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:26:04.946298 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aca15f-ef83-4766-b8c6-05c8d8697c07 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:04.990222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c619e9b-6659-4769-8953-62da96291d62 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:05.040517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:05.119274 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aca15f-ef83-4766-b8c6-05c8d8697c07 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:05.171426 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c619e9b-6659-4769-8953-62da96291d62 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:05.216122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:05.248337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36878b33-dcad-46a6-bc3f-94795e7c1845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:05.249502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36878b33-dcad-46a6-bc3f-94795e7c1845 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:05.249914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36878b33-dcad-46a6-bc3f-94795e7c1845 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:05.250642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36878b33-dcad-46a6-bc3f-94795e7c1845 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:05.257265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac965b6-f59a-449c-a519-4f45a5256904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:05.258296 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac965b6-f59a-449c-a519-4f45a5256904 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:05.258674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac965b6-f59a-449c-a519-4f45a5256904 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:05.259041 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac965b6-f59a-449c-a519-4f45a5256904 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:05.263739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36878b33-dcad-46a6-bc3f-94795e7c1845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:05.265239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36878b33-dcad-46a6-bc3f-94795e7c1845 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:05.275900 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac965b6-f59a-449c-a519-4f45a5256904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-cac965b6-f59a-449c-a519-4f45a5256904 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:05.281284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:05.283075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:05.284624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:05.284624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:05.284624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:05.284624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:05.284624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:05.284624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:05.284624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:05.284624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:05.284624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:05.284624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:05.284624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:05.284624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-36878b33-dcad-46a6-bc3f-94795e7c1845 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:05.285822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:05.296022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:05.299992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:05.299992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:05.299992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:05.299992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 242/485] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:05.299992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16758137-2f51-40dd-a750-4893173f2829 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:05.299992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16758137-2f51-40dd-a750-4893173f2829 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:05.299992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16758137-2f51-40dd-a750-4893173f2829 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:05.299992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 243/486] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:05.299992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d342b-e2e5-4f8f-a17a-03a3c5e37264 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:05.301639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d342b-e2e5-4f8f-a17a-03a3c5e37264 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:05.301639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d342b-e2e5-4f8f-a17a-03a3c5e37264 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:05.302045 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 244/487] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:05.302328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 244/488] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:05.302328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4596c64-5a2d-4ad2-a4e9-146daba77d27 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:05.302328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4596c64-5a2d-4ad2-a4e9-146daba77d27 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:05.302328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4596c64-5a2d-4ad2-a4e9-146daba77d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:05.303454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8b2c-45a0-438c-9555-7bddd17d51a2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:05.303454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8b2c-45a0-438c-9555-7bddd17d51a2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:05.303454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8b2c-45a0-438c-9555-7bddd17d51a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:05.322171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d4596c64-5a2d-4ad2-a4e9-146daba77d27 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:05.322393 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f4bd8b2c-45a0-438c-9555-7bddd17d51a2 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:05.323385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d4596c64-5a2d-4ad2-a4e9-146daba77d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:05.323585 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f4bd8b2c-45a0-438c-9555-7bddd17d51a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:05.422312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 245/489] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:05.422312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4596c64-5a2d-4ad2-a4e9-146daba77d27 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:05.433249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 245/490] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:05.434139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd8b2c-45a0-438c-9555-7bddd17d51a2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:05.515603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:05.609223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd8b2c-45a0-438c-9555-7bddd17d51a2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:05.611281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:05.615405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4596c64-5a2d-4ad2-a4e9-146daba77d27 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:05.652460 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da2dbc-33f7-424e-8fc1-2560c2fab51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:05.653076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da2dbc-33f7-424e-8fc1-2560c2fab51d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:05.653351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da2dbc-33f7-424e-8fc1-2560c2fab51d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:05.653645 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da2dbc-33f7-424e-8fc1-2560c2fab51d None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:05.654325 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da2dbc-33f7-424e-8fc1-2560c2fab51d None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:05.654628 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da2dbc-33f7-424e-8fc1-2560c2fab51d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:05.656043 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da2dbc-33f7-424e-8fc1-2560c2fab51d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:05.666171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 246/491] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:05.672053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da2dbc-33f7-424e-8fc1-2560c2fab51d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:05.699642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a588747-af70-42a4-9241-24628244bab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:05.700164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a588747-af70-42a4-9241-24628244bab7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:05.700433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a588747-af70-42a4-9241-24628244bab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:05.700693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a588747-af70-42a4-9241-24628244bab7 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:05.701348 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a588747-af70-42a4-9241-24628244bab7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:05.701645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a588747-af70-42a4-9241-24628244bab7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:05.703079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a588747-af70-42a4-9241-24628244bab7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:05.735580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 246/492] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:05.746231 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ceef9-118b-4996-8c4c-46eb2801a0f5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:05.747212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ceef9-118b-4996-8c4c-46eb2801a0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:05.747949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ceef9-118b-4996-8c4c-46eb2801a0f5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:05.751017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 247/493] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:05 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:05.752453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ceef9-118b-4996-8c4c-46eb2801a0f5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:05.784754 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da2dbc-33f7-424e-8fc1-2560c2fab51d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:05.844321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9614b6e-33e6-4915-856f-571376ebcfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:05.845599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9614b6e-33e6-4915-856f-571376ebcfc9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:05.846726 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9614b6e-33e6-4915-856f-571376ebcfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:05.847654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9614b6e-33e6-4915-856f-571376ebcfc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:05.868670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9614b6e-33e6-4915-856f-571376ebcfc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:05.871484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9614b6e-33e6-4915-856f-571376ebcfc9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:05.897702 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ceef9-118b-4996-8c4c-46eb2801a0f5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c9614b6e-33e6-4915-856f-571376ebcfc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:05.898584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:05.899781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:05.900694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:05.900694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:05.900694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:05.900694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 247/494] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:05.902768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168e1a7-f190-46a0-a7f0-a7a85a607359 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:05.903256 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168e1a7-f190-46a0-a7f0-a7a85a607359 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:05.903734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168e1a7-f190-46a0-a7f0-a7a85a607359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:05.916356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:05.935008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5168e1a7-f190-46a0-a7f0-a7a85a607359 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:05.937048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5168e1a7-f190-46a0-a7f0-a7a85a607359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:05.940979 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96fa22-fcf4-4927-9b27-385b1a94a007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:05.941648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96fa22-fcf4-4927-9b27-385b1a94a007 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:05.942082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96fa22-fcf4-4927-9b27-385b1a94a007 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:05.942546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96fa22-fcf4-4927-9b27-385b1a94a007 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:05.943542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96fa22-fcf4-4927-9b27-385b1a94a007 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:05.945086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96fa22-fcf4-4927-9b27-385b1a94a007 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:05.956028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:06.018061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 248/495] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:06.018959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168e1a7-f190-46a0-a7f0-a7a85a607359 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:06.069913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 248/496] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:26:05 2022] POST /identity/v3/users => generated 247 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:26:06.071501 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96fa22-fcf4-4927-9b27-385b1a94a007 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:06.187235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168e1a7-f190-46a0-a7f0-a7a85a607359 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:06.268626 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868aed5-6ea0-4200-af32-631377b22abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:06.269606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868aed5-6ea0-4200-af32-631377b22abc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:06.270141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868aed5-6ea0-4200-af32-631377b22abc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:06.270738 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868aed5-6ea0-4200-af32-631377b22abc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:06.294348 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96fa22-fcf4-4927-9b27-385b1a94a007 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:06.296239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868aed5-6ea0-4200-af32-631377b22abc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b868aed5-6ea0-4200-af32-631377b22abc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:06.304089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:06.305948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:06.308440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:06.308440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:06.308440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:06.308440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:06.308440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:06.308440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:06.308440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:06.308440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:06.308440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:06.308440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:06.308440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:06.308440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:06.312401 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 249/497] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:06.312401 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868aed5-6ea0-4200-af32-631377b22abc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:06.320717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:06.343223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b4135-8e8c-4773-9a69-5f742af20a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:06.344334 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b4135-8e8c-4773-9a69-5f742af20a70 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:06.344817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b4135-8e8c-4773-9a69-5f742af20a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:06.345271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b4135-8e8c-4773-9a69-5f742af20a70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:06.364225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b4135-8e8c-4773-9a69-5f742af20a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-940b4135-8e8c-4773-9a69-5f742af20a70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:06.375105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:06.376815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:06.379008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:06.379008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:06.379008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:06.379008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:06.379008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:06.379008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:06.379008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:06.379008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:06.379008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:06.379008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:06.379008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:06.379008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:06.382304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 249/498] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:06.383352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b4135-8e8c-4773-9a69-5f742af20a70 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:06.421009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:06.474905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868aed5-6ea0-4200-af32-631377b22abc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:06.524252 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b4135-8e8c-4773-9a69-5f742af20a70 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:06.529922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9207bc9-8c29-4781-a8ef-134e3d29c743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:06.530572 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9207bc9-8c29-4781-a8ef-134e3d29c743 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:06.530845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9207bc9-8c29-4781-a8ef-134e3d29c743 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:06.531105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9207bc9-8c29-4781-a8ef-134e3d29c743 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:06.531761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9207bc9-8c29-4781-a8ef-134e3d29c743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:06.532124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9207bc9-8c29-4781-a8ef-134e3d29c743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:06.533540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9207bc9-8c29-4781-a8ef-134e3d29c743 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:06.553902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 250/499] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:06.553902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9207bc9-8c29-4781-a8ef-134e3d29c743 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:06.577645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792275e6-8c5f-438c-9f5f-322e4a21d2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:06.578331 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792275e6-8c5f-438c-9f5f-322e4a21d2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:06.578607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792275e6-8c5f-438c-9f5f-322e4a21d2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:06.579398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792275e6-8c5f-438c-9f5f-322e4a21d2c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:06.594986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792275e6-8c5f-438c-9f5f-322e4a21d2c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-792275e6-8c5f-438c-9f5f-322e4a21d2c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:06.605479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:06.606498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:06.607283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:06.607283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:06.607283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:06.607283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:06.607283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:06.607283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:06.607283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:06.607283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:06.607283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:06.607283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:06.607283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:06.607283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:06.607893 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 250/500] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:06.608756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792275e6-8c5f-438c-9f5f-322e4a21d2c5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:06.632547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:06.677982 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9207bc9-8c29-4781-a8ef-134e3d29c743 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:06.719153 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3efd2-02b0-4d20-9dda-95000a534e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:06.719772 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3efd2-02b0-4d20-9dda-95000a534e41 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:06.720050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3efd2-02b0-4d20-9dda-95000a534e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:06.720325 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3efd2-02b0-4d20-9dda-95000a534e41 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:06.721048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3efd2-02b0-4d20-9dda-95000a534e41 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:06.721355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3efd2-02b0-4d20-9dda-95000a534e41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:06.722928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3efd2-02b0-4d20-9dda-95000a534e41 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:06.734055 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:06.738616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 251/501] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:06.739743 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3efd2-02b0-4d20-9dda-95000a534e41 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:06.753687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792275e6-8c5f-438c-9f5f-322e4a21d2c5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:06.798093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331c758-5f71-4aee-9a4e-fe708287d019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:06.798738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331c758-5f71-4aee-9a4e-fe708287d019 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:06.799533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331c758-5f71-4aee-9a4e-fe708287d019 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:06.799798 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331c758-5f71-4aee-9a4e-fe708287d019 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:06.800466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331c758-5f71-4aee-9a4e-fe708287d019 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:06.800770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331c758-5f71-4aee-9a4e-fe708287d019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:06.802181 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331c758-5f71-4aee-9a4e-fe708287d019 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:06.812355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 251/502] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:06.813679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331c758-5f71-4aee-9a4e-fe708287d019 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:06.871229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3efd2-02b0-4d20-9dda-95000a534e41 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:06.907200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:06.925223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d7a9f-2f76-4d75-a9d4-a5ef2b616259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:06.926474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d7a9f-2f76-4d75-a9d4-a5ef2b616259 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:06.927026 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d7a9f-2f76-4d75-a9d4-a5ef2b616259 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:06.927499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d7a9f-2f76-4d75-a9d4-a5ef2b616259 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:06.940265 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d7a9f-2f76-4d75-a9d4-a5ef2b616259 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:06.942032 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d7a9f-2f76-4d75-a9d4-a5ef2b616259 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:06.957753 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331c758-5f71-4aee-9a4e-fe708287d019 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5e6d7a9f-2f76-4d75-a9d4-a5ef2b616259 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:06.957967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:06.959503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:06.961190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:06.961190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:06.961190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:06.961190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:06.961190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:06.961190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:06.961190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:06.961190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:06.961190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 08:26:06.961190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:06.965630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 252/503] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:26:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:06.966358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d7a9f-2f76-4d75-a9d4-a5ef2b616259 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:07.003801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738994c-6aee-4c03-a71b-dddb36587f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:07.004571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738994c-6aee-4c03-a71b-dddb36587f67 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:07.004922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738994c-6aee-4c03-a71b-dddb36587f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:07.005305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738994c-6aee-4c03-a71b-dddb36587f67 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:07.006048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738994c-6aee-4c03-a71b-dddb36587f67 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:07.006463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738994c-6aee-4c03-a71b-dddb36587f67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:07.007996 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738994c-6aee-4c03-a71b-dddb36587f67 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:07.017223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 252/504] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:07.018170 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738994c-6aee-4c03-a71b-dddb36587f67 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:07.038250 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:07.104562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d7a9f-2f76-4d75-a9d4-a5ef2b616259 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:07.142904 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12193c-670b-4d26-b8e8-10396c1d65bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:07.143558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12193c-670b-4d26-b8e8-10396c1d65bc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:07.143830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12193c-670b-4d26-b8e8-10396c1d65bc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:07.144099 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12193c-670b-4d26-b8e8-10396c1d65bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:07.148056 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738994c-6aee-4c03-a71b-dddb36587f67 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:07.156797 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12193c-670b-4d26-b8e8-10396c1d65bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:07.158864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12193c-670b-4d26-b8e8-10396c1d65bc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-aa12193c-670b-4d26-b8e8-10396c1d65bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:07.177843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:07.178883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:07.180010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:07.180010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:07.180010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:07.180010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 253/505] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:07.180010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12193c-670b-4d26-b8e8-10396c1d65bc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:07.196833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56b869-9755-4d8e-948a-43bad886278d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:07.197932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56b869-9755-4d8e-948a-43bad886278d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:07.199221 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56b869-9755-4d8e-948a-43bad886278d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:07.199221 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56b869-9755-4d8e-948a-43bad886278d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:07.220075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56b869-9755-4d8e-948a-43bad886278d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:07.222057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56b869-9755-4d8e-948a-43bad886278d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:07.234679 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9e56b869-9755-4d8e-948a-43bad886278d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:07.245133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:07.250264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:07.251170 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:07.251170 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:07.251170 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:07.251170 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 253/506] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:07.251170 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56b869-9755-4d8e-948a-43bad886278d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:07.328890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12193c-670b-4d26-b8e8-10396c1d65bc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:07.334580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:07.377488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720b826-16ef-41a9-a416-c673608560d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:07.378004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720b826-16ef-41a9-a416-c673608560d3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:07.378766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720b826-16ef-41a9-a416-c673608560d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:07.379097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720b826-16ef-41a9-a416-c673608560d3 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:07.379991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720b826-16ef-41a9-a416-c673608560d3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:07.380297 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720b826-16ef-41a9-a416-c673608560d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:07.381723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720b826-16ef-41a9-a416-c673608560d3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:07.386490 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56b869-9755-4d8e-948a-43bad886278d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:07.426460 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 254/507] 10.222.0.44 () {56 vars in 1137 bytes} [Sun Sep 11 08:26:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:07.427388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720b826-16ef-41a9-a416-c673608560d3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:07.439969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0351ec-fa28-4bfa-a8d3-403674e30bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:07.440716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0351ec-fa28-4bfa-a8d3-403674e30bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:07.441074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0351ec-fa28-4bfa-a8d3-403674e30bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:07.441397 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0351ec-fa28-4bfa-a8d3-403674e30bc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:07.455795 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0351ec-fa28-4bfa-a8d3-403674e30bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:07.457269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0351ec-fa28-4bfa-a8d3-403674e30bc6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0e0351ec-fa28-4bfa-a8d3-403674e30bc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:07.473739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:07.474831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:07.475660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:07.475660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:07.475660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:07.475660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 254/508] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:07.476031 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0351ec-fa28-4bfa-a8d3-403674e30bc6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:07.601761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:07.604924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0351ec-fa28-4bfa-a8d3-403674e30bc6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:07.611547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720b826-16ef-41a9-a416-c673608560d3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:07.627946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:07.654307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f1285-a5dd-4a63-a6f7-15270d872e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:07.654962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f1285-a5dd-4a63-a6f7-15270d872e0b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:07.655238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f1285-a5dd-4a63-a6f7-15270d872e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:07.655512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f1285-a5dd-4a63-a6f7-15270d872e0b None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:07.656160 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f1285-a5dd-4a63-a6f7-15270d872e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:07.656461 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f1285-a5dd-4a63-a6f7-15270d872e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:07.657998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f1285-a5dd-4a63-a6f7-15270d872e0b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:07.666284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1f389-6814-46e9-b114-5b133a2673cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:07.666895 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1f389-6814-46e9-b114-5b133a2673cb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:07.667168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1f389-6814-46e9-b114-5b133a2673cb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:07.667434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1f389-6814-46e9-b114-5b133a2673cb None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:07.668293 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1f389-6814-46e9-b114-5b133a2673cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:07.668456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1f389-6814-46e9-b114-5b133a2673cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:07.670340 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1f389-6814-46e9-b114-5b133a2673cb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:07.685402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 255/509] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:07.685402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f1285-a5dd-4a63-a6f7-15270d872e0b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:07.691326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 255/510] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:07.691326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1f389-6814-46e9-b114-5b133a2673cb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:07.802248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f1285-a5dd-4a63-a6f7-15270d872e0b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:07.824946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1f389-6814-46e9-b114-5b133a2673cb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:07.848407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2fed1-4ee6-4ffb-9a9f-8475eac23837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:07.849127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2fed1-4ee6-4ffb-9a9f-8475eac23837 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:07.849395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2fed1-4ee6-4ffb-9a9f-8475eac23837 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:07.849777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2fed1-4ee6-4ffb-9a9f-8475eac23837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:07.882496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80ff3e-c749-46a0-882a-1975c708ef44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:07.882496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80ff3e-c749-46a0-882a-1975c708ef44 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:07.882723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80ff3e-c749-46a0-882a-1975c708ef44 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:07.882989 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80ff3e-c749-46a0-882a-1975c708ef44 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:07.883663 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80ff3e-c749-46a0-882a-1975c708ef44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:07.883939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80ff3e-c749-46a0-882a-1975c708ef44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:07.885328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80ff3e-c749-46a0-882a-1975c708ef44 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:07.904919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 256/511] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:07.904919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80ff3e-c749-46a0-882a-1975c708ef44 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:07.917922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:07.927646 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:07.946335 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2fed1-4ee6-4ffb-9a9f-8475eac23837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:07.948205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2fed1-4ee6-4ffb-9a9f-8475eac23837 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:08.035336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80ff3e-c749-46a0-882a-1975c708ef44 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:08.087039 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 256/512] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:26:08.088254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2fed1-4ee6-4ffb-9a9f-8475eac23837 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:08.093345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e9bad-ddcc-4825-be43-2dc65c371ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:08.093588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e9bad-ddcc-4825-be43-2dc65c371ceb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:08.093821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e9bad-ddcc-4825-be43-2dc65c371ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:08.094088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e9bad-ddcc-4825-be43-2dc65c371ceb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:08.109811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e9bad-ddcc-4825-be43-2dc65c371ceb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:08.111356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e9bad-ddcc-4825-be43-2dc65c371ceb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-910e9bad-ddcc-4825-be43-2dc65c371ceb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:08.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:08.138732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:08.143548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:08.143548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:08.143548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:08.143548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:08.143548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:08.143548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:08.143548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:08.143548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:08.143548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 08:26:08.143548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:08.143548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 257/513] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:26:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:08.143548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e9bad-ddcc-4825-be43-2dc65c371ceb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:08.198972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:08.228529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2fed1-4ee6-4ffb-9a9f-8475eac23837 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:08.254190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:08.269055 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b42c6-8cec-4b5e-af3d-e71801bb6453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:08.269695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b42c6-8cec-4b5e-af3d-e71801bb6453 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:08.269980 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b42c6-8cec-4b5e-af3d-e71801bb6453 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:08.270277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b42c6-8cec-4b5e-af3d-e71801bb6453 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:08.278556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b42c6-8cec-4b5e-af3d-e71801bb6453 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:08.280060 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b42c6-8cec-4b5e-af3d-e71801bb6453 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:08.281027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e9bad-ddcc-4825-be43-2dc65c371ceb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-667b42c6-8cec-4b5e-af3d-e71801bb6453 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:08.296385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:08.297560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:08.298710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:08.298710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:08.298710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:08.298710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:08.298710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:08.298710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:08.298710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:08.298710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:08.298710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:26:08.298710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:08.305964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 257/514] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 08:26:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:08.307228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b42c6-8cec-4b5e-af3d-e71801bb6453 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:08.327704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84644c-54d6-4eda-84dc-455c048c1f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:08.328354 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84644c-54d6-4eda-84dc-455c048c1f0b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:08.328629 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84644c-54d6-4eda-84dc-455c048c1f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:08.328903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84644c-54d6-4eda-84dc-455c048c1f0b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:08.337896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84644c-54d6-4eda-84dc-455c048c1f0b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:08.339479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84644c-54d6-4eda-84dc-455c048c1f0b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1e84644c-54d6-4eda-84dc-455c048c1f0b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:08.355042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:08.364095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:08.365911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:08.365911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:08.365911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:08.365911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:08.365911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:08.365911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:08.365911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:08.365911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:08.365911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 08:26:08.365911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:08.365911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 258/515] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:08.365911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84644c-54d6-4eda-84dc-455c048c1f0b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:08.433373 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b42c6-8cec-4b5e-af3d-e71801bb6453 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:08.478711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffe95a-f58f-4c74-b8b2-02cb9b54ad2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:08.479570 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffe95a-f58f-4c74-b8b2-02cb9b54ad2f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:08.480082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffe95a-f58f-4c74-b8b2-02cb9b54ad2f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:08.480486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffe95a-f58f-4c74-b8b2-02cb9b54ad2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:08.482782 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:08.492884 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffe95a-f58f-4c74-b8b2-02cb9b54ad2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:08.497767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffe95a-f58f-4c74-b8b2-02cb9b54ad2f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:08.497921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84644c-54d6-4eda-84dc-455c048c1f0b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-50ffe95a-f58f-4c74-b8b2-02cb9b54ad2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:08.509494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:08.510583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:08.511512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:08.511512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:08.511512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:08.511512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 258/516] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:08.511512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffe95a-f58f-4c74-b8b2-02cb9b54ad2f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:08.537716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fc326-f407-4366-be7c-d3618ec01ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:08.538390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fc326-f407-4366-be7c-d3618ec01ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:08.538537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fc326-f407-4366-be7c-d3618ec01ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:08.538800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fc326-f407-4366-be7c-d3618ec01ce8 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:08.539526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8fc326-f407-4366-be7c-d3618ec01ce8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:08.539820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8fc326-f407-4366-be7c-d3618ec01ce8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:08.541296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8fc326-f407-4366-be7c-d3618ec01ce8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:08.544261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:08.559725 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 259/517] 10.222.0.44 () {56 vars in 1137 bytes} [Sun Sep 11 08:26:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:08.560568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fc326-f407-4366-be7c-d3618ec01ce8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:08.649196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffe95a-f58f-4c74-b8b2-02cb9b54ad2f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:08.691473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fc326-f407-4366-be7c-d3618ec01ce8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:08.694793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c560f-36a2-4aef-913f-2e525f6ab97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:08.695412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c560f-36a2-4aef-913f-2e525f6ab97e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:08.695685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c560f-36a2-4aef-913f-2e525f6ab97e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:08.695950 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c560f-36a2-4aef-913f-2e525f6ab97e None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:08.696685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c560f-36a2-4aef-913f-2e525f6ab97e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:08.696983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c560f-36a2-4aef-913f-2e525f6ab97e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:08.702187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c560f-36a2-4aef-913f-2e525f6ab97e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:08.734748 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db9c72-38e5-42c9-8fa2-da0d4a0a17e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:08.735366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db9c72-38e5-42c9-8fa2-da0d4a0a17e3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:08.735637 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db9c72-38e5-42c9-8fa2-da0d4a0a17e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:08.735905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db9c72-38e5-42c9-8fa2-da0d4a0a17e3 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:08.737224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db9c72-38e5-42c9-8fa2-da0d4a0a17e3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:08.737525 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db9c72-38e5-42c9-8fa2-da0d4a0a17e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:08.739026 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db9c72-38e5-42c9-8fa2-da0d4a0a17e3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:08.743682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 259/518] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 08:26:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:08.745882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddf67b-9422-4f68-9b68-c5c7653d59ac None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:08.746394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddf67b-9422-4f68-9b68-c5c7653d59ac None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:08.746864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddf67b-9422-4f68-9b68-c5c7653d59ac None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:08.750232 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 260/519] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:08.751272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddf67b-9422-4f68-9b68-c5c7653d59ac None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:08.765200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 260/520] 10.222.0.44 () {56 vars in 1139 bytes} [Sun Sep 11 08:26:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:08.766215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db9c72-38e5-42c9-8fa2-da0d4a0a17e3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:08.807784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:08.827419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:08.873031 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddf67b-9422-4f68-9b68-c5c7653d59ac None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:08.885295 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db9c72-38e5-42c9-8fa2-da0d4a0a17e3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:08.912992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e95fea-a96f-4932-81bf-d8ca9a5afabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:08.913613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e95fea-a96f-4932-81bf-d8ca9a5afabe None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:08.913886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e95fea-a96f-4932-81bf-d8ca9a5afabe None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:08.914174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e95fea-a96f-4932-81bf-d8ca9a5afabe None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:08.914831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e95fea-a96f-4932-81bf-d8ca9a5afabe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:08.915129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e95fea-a96f-4932-81bf-d8ca9a5afabe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:08.916504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e95fea-a96f-4932-81bf-d8ca9a5afabe None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:08.929366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 261/521] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:08.930250 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e95fea-a96f-4932-81bf-d8ca9a5afabe None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:08.935842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba579e-9b47-44ee-b8f5-047c7e368d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:08.936890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba579e-9b47-44ee-b8f5-047c7e368d61 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:08.937313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba579e-9b47-44ee-b8f5-047c7e368d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:08.937783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba579e-9b47-44ee-b8f5-047c7e368d61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:09.037567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba579e-9b47-44ee-b8f5-047c7e368d61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:09.039447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba579e-9b47-44ee-b8f5-047c7e368d61 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:09.044968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e95fea-a96f-4932-81bf-d8ca9a5afabe None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:09.047429 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:09.088220 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeec312-4859-410e-b977-da261489c54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:09.088836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeec312-4859-410e-b977-da261489c54f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:09.089114 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeec312-4859-410e-b977-da261489c54f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:09.089385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeec312-4859-410e-b977-da261489c54f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:09.101597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:09.156446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 261/522] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:26:09.157724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741bb7a-e84a-4815-8486-5f0e948cfed0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:09.158004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741bb7a-e84a-4815-8486-5f0e948cfed0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:09.158877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741bb7a-e84a-4815-8486-5f0e948cfed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:09.176970 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4741bb7a-e84a-4815-8486-5f0e948cfed0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:09.178234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4741bb7a-e84a-4815-8486-5f0e948cfed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:09.196413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeec312-4859-410e-b977-da261489c54f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:09.197940 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeec312-4859-410e-b977-da261489c54f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:09.282216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 262/523] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:09.282216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741bb7a-e84a-4815-8486-5f0e948cfed0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:09.353864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 262/524] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:26:09.355069 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeec312-4859-410e-b977-da261489c54f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:09.363713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:09.428247 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:09.470408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741bb7a-e84a-4815-8486-5f0e948cfed0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:09.517318 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeec312-4859-410e-b977-da261489c54f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:09.532490 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad535-28b0-4498-b067-08a727a5aaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:09.533350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad535-28b0-4498-b067-08a727a5aaa2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:09.533752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad535-28b0-4498-b067-08a727a5aaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:09.534233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad535-28b0-4498-b067-08a727a5aaa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:09.586215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2865296-dce2-49f4-8aa1-0c84924ed35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:09.587238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2865296-dce2-49f4-8aa1-0c84924ed35c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:09.587678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2865296-dce2-49f4-8aa1-0c84924ed35c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:09.588113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2865296-dce2-49f4-8aa1-0c84924ed35c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:09.596953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2865296-dce2-49f4-8aa1-0c84924ed35c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:09.598559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2865296-dce2-49f4-8aa1-0c84924ed35c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d2865296-dce2-49f4-8aa1-0c84924ed35c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:09.613294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:09.615227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:09.617144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:09.617144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:09.617144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:09.617144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:09.617144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:09.617144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:09.617144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:09.617144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:09.617144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 263/525] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:09.617144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2865296-dce2-49f4-8aa1-0c84924ed35c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:09.635720 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ad535-28b0-4498-b067-08a727a5aaa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:09.638113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ad535-28b0-4498-b067-08a727a5aaa2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:09.718669 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:09.729879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:09.749778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 263/526] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:26:09.750681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad535-28b0-4498-b067-08a727a5aaa2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:09.787529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2865296-dce2-49f4-8aa1-0c84924ed35c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:09.824908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ee4e1-7a56-45f1-8249-52c7cccb8922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:09.825536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ee4e1-7a56-45f1-8249-52c7cccb8922 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:09.825833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ee4e1-7a56-45f1-8249-52c7cccb8922 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:09.826078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ee4e1-7a56-45f1-8249-52c7cccb8922 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:09.837205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ee4e1-7a56-45f1-8249-52c7cccb8922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:09.838730 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ee4e1-7a56-45f1-8249-52c7cccb8922 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f47ee4e1-7a56-45f1-8249-52c7cccb8922 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:09.855815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:09.857597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:09.859337 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:09.859337 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:09.859337 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:09.859337 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 264/527] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:09.859337 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ee4e1-7a56-45f1-8249-52c7cccb8922 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:09.898389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad535-28b0-4498-b067-08a727a5aaa2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:09.935218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641f82b-5281-42c3-b3ad-41481f81bd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:09.935849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641f82b-5281-42c3-b3ad-41481f81bd16 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:09.936123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641f82b-5281-42c3-b3ad-41481f81bd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:09.936389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641f82b-5281-42c3-b3ad-41481f81bd16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:09.947938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641f82b-5281-42c3-b3ad-41481f81bd16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:09.949440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641f82b-5281-42c3-b3ad-41481f81bd16 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1641f82b-5281-42c3-b3ad-41481f81bd16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:09.965461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:09.966841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:09.968501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:09.968501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:09.968501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:09.968501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 264/528] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:09.968501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641f82b-5281-42c3-b3ad-41481f81bd16 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:10.011940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:10.020291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ee4e1-7a56-45f1-8249-52c7cccb8922 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:10.037725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:10.059460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c4bed-df69-4a38-9c55-65617c369658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:10.060097 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c4bed-df69-4a38-9c55-65617c369658 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:10.060370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c4bed-df69-4a38-9c55-65617c369658 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:10.060619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c4bed-df69-4a38-9c55-65617c369658 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:10.073178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1c4bed-df69-4a38-9c55-65617c369658 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1b1c4bed-df69-4a38-9c55-65617c369658 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:10.078396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:10.080112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 265/529] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:10.081685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c4bed-df69-4a38-9c55-65617c369658 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:10.088418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641f82b-5281-42c3-b3ad-41481f81bd16 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:10.131552 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f062a-8b34-49bf-8033-476e0b33abf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:10.132233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f062a-8b34-49bf-8033-476e0b33abf8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:10.132594 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f062a-8b34-49bf-8033-476e0b33abf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:10.132949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f062a-8b34-49bf-8033-476e0b33abf8 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:10.133785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f062a-8b34-49bf-8033-476e0b33abf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:10.134261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f062a-8b34-49bf-8033-476e0b33abf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:10.135879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f062a-8b34-49bf-8033-476e0b33abf8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:10.156101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 265/530] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:10.156995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f062a-8b34-49bf-8033-476e0b33abf8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:10.206895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c4bed-df69-4a38-9c55-65617c369658 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:10.245031 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a342a6b-6aa1-4384-a04f-288f09fc1eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:10.245666 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a342a6b-6aa1-4384-a04f-288f09fc1eab None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:10.245941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a342a6b-6aa1-4384-a04f-288f09fc1eab None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:10.246243 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a342a6b-6aa1-4384-a04f-288f09fc1eab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:10.257650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a342a6b-6aa1-4384-a04f-288f09fc1eab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:10.259160 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a342a6b-6aa1-4384-a04f-288f09fc1eab None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:10.262066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:10.269436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f062a-8b34-49bf-8033-476e0b33abf8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6a342a6b-6aa1-4384-a04f-288f09fc1eab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:10.274519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:10.275497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:10.276340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:10.276340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:10.276340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:10.276340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 266/531] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:10.276556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a342a6b-6aa1-4384-a04f-288f09fc1eab None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:10.312457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:10.312736 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df9377-3103-473b-80f9-37f02d4b963a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:10.313322 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df9377-3103-473b-80f9-37f02d4b963a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:10.313603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df9377-3103-473b-80f9-37f02d4b963a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:10.313879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df9377-3103-473b-80f9-37f02d4b963a None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:10.314562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df9377-3103-473b-80f9-37f02d4b963a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:10.314865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df9377-3103-473b-80f9-37f02d4b963a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:10.316305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df9377-3103-473b-80f9-37f02d4b963a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:10.328677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 266/532] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:10.329566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df9377-3103-473b-80f9-37f02d4b963a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:10.394662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a342a6b-6aa1-4384-a04f-288f09fc1eab None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:10.447141 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586a143-a6aa-4c9a-8eec-73ae994a4cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:10.448091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586a143-a6aa-4c9a-8eec-73ae994a4cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:10.448541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586a143-a6aa-4c9a-8eec-73ae994a4cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:10.448988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586a143-a6aa-4c9a-8eec-73ae994a4cd8 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:10.450069 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586a143-a6aa-4c9a-8eec-73ae994a4cd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:10.450486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586a143-a6aa-4c9a-8eec-73ae994a4cd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:10.452040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586a143-a6aa-4c9a-8eec-73ae994a4cd8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:10.460259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df9377-3103-473b-80f9-37f02d4b963a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:10.466222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 267/533] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:10.467119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586a143-a6aa-4c9a-8eec-73ae994a4cd8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:10.504329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e7967-7eee-4e41-85c4-607ed052a7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:10.504975 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e7967-7eee-4e41-85c4-607ed052a7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:10.505295 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e7967-7eee-4e41-85c4-607ed052a7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:10.505595 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e7967-7eee-4e41-85c4-607ed052a7c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:10.506355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e7967-7eee-4e41-85c4-607ed052a7c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:10.506703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e7967-7eee-4e41-85c4-607ed052a7c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:10.508170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e7967-7eee-4e41-85c4-607ed052a7c2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:10.519871 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 267/534] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:10.520870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e7967-7eee-4e41-85c4-607ed052a7c2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:10.571799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:10.584425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:10.609439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586a143-a6aa-4c9a-8eec-73ae994a4cd8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:10.650205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec5102-a3ae-4d02-b933-8d7165f4650d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:10.650724 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec5102-a3ae-4d02-b933-8d7165f4650d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:10.650990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec5102-a3ae-4d02-b933-8d7165f4650d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:10.651273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec5102-a3ae-4d02-b933-8d7165f4650d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:10.652023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec5102-a3ae-4d02-b933-8d7165f4650d None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=434c9e87bda343659d2f410a6cdbf805) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:10.652326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec5102-a3ae-4d02-b933-8d7165f4650d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:10.653800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec5102-a3ae-4d02-b933-8d7165f4650d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:10.660709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 268/535] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:10 2022] GET /identity/v3/role_assignments?user.id=434c9e87bda343659d2f410a6cdbf805&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:10.661601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec5102-a3ae-4d02-b933-8d7165f4650d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:10.702399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e7967-7eee-4e41-85c4-607ed052a7c2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:10.752881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a8b10-3530-42bf-97d2-dcf8d0dedce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:10.754617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a8b10-3530-42bf-97d2-dcf8d0dedce9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:10.755052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a8b10-3530-42bf-97d2-dcf8d0dedce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:10.755686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a8b10-3530-42bf-97d2-dcf8d0dedce9 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:10.756530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a8b10-3530-42bf-97d2-dcf8d0dedce9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:10.757061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a8b10-3530-42bf-97d2-dcf8d0dedce9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:10.758981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a8b10-3530-42bf-97d2-dcf8d0dedce9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:10.776392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 268/536] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:10.777849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a8b10-3530-42bf-97d2-dcf8d0dedce9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:10.811894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec5102-a3ae-4d02-b933-8d7165f4650d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:10.921759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e966175-c9f8-428d-b926-cc5a937a101e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:10.922840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e966175-c9f8-428d-b926-cc5a937a101e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:10.923361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e966175-c9f8-428d-b926-cc5a937a101e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:10.923927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e966175-c9f8-428d-b926-cc5a937a101e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:10.928864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:10.953355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e966175-c9f8-428d-b926-cc5a937a101e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:10.955583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e966175-c9f8-428d-b926-cc5a937a101e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5e966175-c9f8-428d-b926-cc5a937a101e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:10.968833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:10.970648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:10.972284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:10.972284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:10.972284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:10.972284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:10.972284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:10.972284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:10.972284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:10.972284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:10.972284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 269/537] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:10.972284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e966175-c9f8-428d-b926-cc5a937a101e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:10.989957 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a8b10-3530-42bf-97d2-dcf8d0dedce9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:11.005852 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:11.043921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab6827-a8b7-48e4-afe4-7ba52d20beda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:11.044571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab6827-a8b7-48e4-afe4-7ba52d20beda None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:11.044854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab6827-a8b7-48e4-afe4-7ba52d20beda None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:11.045119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab6827-a8b7-48e4-afe4-7ba52d20beda None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:11.053996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab6827-a8b7-48e4-afe4-7ba52d20beda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:11.055503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab6827-a8b7-48e4-afe4-7ba52d20beda None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7fab6827-a8b7-48e4-afe4-7ba52d20beda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:11.070254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:11.071284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:11.072229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:11.072229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:11.072229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:11.072229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:11.072229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:11.072229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:11.072229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:11.072229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:11.076376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 269/538] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:11.077337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab6827-a8b7-48e4-afe4-7ba52d20beda None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:11.145215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e966175-c9f8-428d-b926-cc5a937a101e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:11.217738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d443f4d-a54c-4b94-bdcf-b66703a82754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:11.219248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d443f4d-a54c-4b94-bdcf-b66703a82754 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:11.219740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d443f4d-a54c-4b94-bdcf-b66703a82754 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:11.220720 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d443f4d-a54c-4b94-bdcf-b66703a82754 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:11.239054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d443f4d-a54c-4b94-bdcf-b66703a82754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:11.240556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d443f4d-a54c-4b94-bdcf-b66703a82754 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:11.260619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab6827-a8b7-48e4-afe4-7ba52d20beda None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7d443f4d-a54c-4b94-bdcf-b66703a82754 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:11.266213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:11.267169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:11.268063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:11.268063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:11.268063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:11.271703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 270/539] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:11.271703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d443f4d-a54c-4b94-bdcf-b66703a82754 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:11.327465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604d2e7-f779-48f8-89db-75bb326d5299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:11.328575 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604d2e7-f779-48f8-89db-75bb326d5299 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:11.329184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604d2e7-f779-48f8-89db-75bb326d5299 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:11.329650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604d2e7-f779-48f8-89db-75bb326d5299 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:11.335200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:11.343246 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:11.346354 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604d2e7-f779-48f8-89db-75bb326d5299 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:11.349178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604d2e7-f779-48f8-89db-75bb326d5299 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5604d2e7-f779-48f8-89db-75bb326d5299 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:11.373455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:11.374733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:11.375660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:11.375660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:11.375660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:11.375660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:11.375660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:11.375660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:11.375660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:11.375660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:11.379696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 270/540] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:11.381355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604d2e7-f779-48f8-89db-75bb326d5299 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:11.389945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d443f4d-a54c-4b94-bdcf-b66703a82754 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:11.427215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dcbb9-5c25-4196-a1ae-d2c7090bb953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:11.427821 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dcbb9-5c25-4196-a1ae-d2c7090bb953 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:11.428098 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dcbb9-5c25-4196-a1ae-d2c7090bb953 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:11.428376 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dcbb9-5c25-4196-a1ae-d2c7090bb953 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:11.429076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dcbb9-5c25-4196-a1ae-d2c7090bb953 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:11.429381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dcbb9-5c25-4196-a1ae-d2c7090bb953 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:11.430932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dcbb9-5c25-4196-a1ae-d2c7090bb953 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:11.455605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 271/541] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:11.455605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dcbb9-5c25-4196-a1ae-d2c7090bb953 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:11.552114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604d2e7-f779-48f8-89db-75bb326d5299 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:11.609767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966aa5e-e9cb-4573-962f-e7b779b5ad55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:11.610782 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966aa5e-e9cb-4573-962f-e7b779b5ad55 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:11.611800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966aa5e-e9cb-4573-962f-e7b779b5ad55 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:11.612264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966aa5e-e9cb-4573-962f-e7b779b5ad55 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:11.613448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966aa5e-e9cb-4573-962f-e7b779b5ad55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:11.613972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966aa5e-e9cb-4573-962f-e7b779b5ad55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:11.616605 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966aa5e-e9cb-4573-962f-e7b779b5ad55 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:11.617328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dcbb9-5c25-4196-a1ae-d2c7090bb953 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:11.637003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 271/542] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:11.638172 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966aa5e-e9cb-4573-962f-e7b779b5ad55 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:11.664453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:11.683033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50dee9-e9bb-4b44-a18b-314ec1c7a1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:11.684024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50dee9-e9bb-4b44-a18b-314ec1c7a1fc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:11.684474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50dee9-e9bb-4b44-a18b-314ec1c7a1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:11.684902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab50dee9-e9bb-4b44-a18b-314ec1c7a1fc None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:11.686003 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50dee9-e9bb-4b44-a18b-314ec1c7a1fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:11.686611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50dee9-e9bb-4b44-a18b-314ec1c7a1fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:11.689385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab50dee9-e9bb-4b44-a18b-314ec1c7a1fc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:11.714556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 272/543] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:11.714556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50dee9-e9bb-4b44-a18b-314ec1c7a1fc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:11.737360 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:11.804543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966aa5e-e9cb-4573-962f-e7b779b5ad55 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:11.851597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc656f-7a1a-4e82-bc54-129490e05cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:11.852501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc656f-7a1a-4e82-bc54-129490e05cff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:11.852941 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc656f-7a1a-4e82-bc54-129490e05cff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:11.853207 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc656f-7a1a-4e82-bc54-129490e05cff None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:11.853992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc656f-7a1a-4e82-bc54-129490e05cff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:11.854898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc656f-7a1a-4e82-bc54-129490e05cff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:11.855751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc656f-7a1a-4e82-bc54-129490e05cff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:11.863319 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab50dee9-e9bb-4b44-a18b-314ec1c7a1fc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:11.885076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 272/544] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:11.886310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc656f-7a1a-4e82-bc54-129490e05cff None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:11.922141 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ea943-0802-46a8-aa35-c49dc8a77275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:11.923024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ea943-0802-46a8-aa35-c49dc8a77275 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:11.923305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ea943-0802-46a8-aa35-c49dc8a77275 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:11.923642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ea943-0802-46a8-aa35-c49dc8a77275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:11.924958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ea943-0802-46a8-aa35-c49dc8a77275 None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=7b86978df61245ae88fead154193fcd4) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:11.925271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ea943-0802-46a8-aa35-c49dc8a77275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:11.926845 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ea943-0802-46a8-aa35-c49dc8a77275 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:11.942176 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 273/545] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:11 2022] GET /identity/v3/role_assignments?user.id=7b86978df61245ae88fead154193fcd4&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:11.942176 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ea943-0802-46a8-aa35-c49dc8a77275 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:12.046047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:12.067527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc656f-7a1a-4e82-bc54-129490e05cff None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:12.107263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:12.146809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be42215-5dcc-4d98-a6b1-2c35408a101e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:12.148755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be42215-5dcc-4d98-a6b1-2c35408a101e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:12.150444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be42215-5dcc-4d98-a6b1-2c35408a101e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:12.151270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be42215-5dcc-4d98-a6b1-2c35408a101e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:12.153239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be42215-5dcc-4d98-a6b1-2c35408a101e None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=58c8502b91ea4ba79e85818727422410) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:12.154122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be42215-5dcc-4d98-a6b1-2c35408a101e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:12.157220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be42215-5dcc-4d98-a6b1-2c35408a101e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:12.162332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ea943-0802-46a8-aa35-c49dc8a77275 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:12.180397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 273/546] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:11 2022] GET /identity/v3/role_assignments?user.id=58c8502b91ea4ba79e85818727422410&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:12.182504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be42215-5dcc-4d98-a6b1-2c35408a101e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:12.238759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f8e53-8ba2-4fd3-a651-dd34a6ddea07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:12.240403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f8e53-8ba2-4fd3-a651-dd34a6ddea07 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:12.241194 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f8e53-8ba2-4fd3-a651-dd34a6ddea07 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:12.241913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f8e53-8ba2-4fd3-a651-dd34a6ddea07 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:12.257521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f8e53-8ba2-4fd3-a651-dd34a6ddea07 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:12.260718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f8e53-8ba2-4fd3-a651-dd34a6ddea07 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e56f8e53-8ba2-4fd3-a651-dd34a6ddea07 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:12.302185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:12.304029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:12.305866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:12.305866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:12.305866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:12.305866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:12.305866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:12.305866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:12.305866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:12.305866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:12.305866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 08:26:12.305866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:12.312736 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 274/547] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:26:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:12.318208 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f8e53-8ba2-4fd3-a651-dd34a6ddea07 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:12.408849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be42215-5dcc-4d98-a6b1-2c35408a101e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:12.486315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0b07a-baa9-48f1-b390-f288e7568ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:12.486988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0b07a-baa9-48f1-b390-f288e7568ced None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:12.487308 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0b07a-baa9-48f1-b390-f288e7568ced None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:12.487552 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0b07a-baa9-48f1-b390-f288e7568ced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:12.488272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0b07a-baa9-48f1-b390-f288e7568ced None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=92867e15dcf649adad78bf7a29a996f9) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:12.488581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0b07a-baa9-48f1-b390-f288e7568ced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:12.490045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0b07a-baa9-48f1-b390-f288e7568ced None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:12.496861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 274/548] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:12 2022] GET /identity/v3/role_assignments?user.id=92867e15dcf649adad78bf7a29a996f9&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:12.568981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f8e53-8ba2-4fd3-a651-dd34a6ddea07 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:12.589762 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:12.644723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a0685-0657-40fa-ba80-2a2775faffcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:12.647122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a0685-0657-40fa-ba80-2a2775faffcb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:12.647934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a0685-0657-40fa-ba80-2a2775faffcb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:12.648697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a0685-0657-40fa-ba80-2a2775faffcb None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:12.650496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a0685-0657-40fa-ba80-2a2775faffcb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:12.651435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a0685-0657-40fa-ba80-2a2775faffcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:12.654863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a0685-0657-40fa-ba80-2a2775faffcb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:12.692248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 275/549] 10.222.0.44 () {56 vars in 1135 bytes} [Sun Sep 11 08:26:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:12.702186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0b07a-baa9-48f1-b390-f288e7568ced None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:12.820795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:12.960359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0b07a-baa9-48f1-b390-f288e7568ced None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:13.040955 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d7b96-ec3b-4309-a741-d772b7077399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:13.042014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d7b96-ec3b-4309-a741-d772b7077399 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:13.042538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d7b96-ec3b-4309-a741-d772b7077399 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:13.043031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d7b96-ec3b-4309-a741-d772b7077399 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:13.194605 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d7b96-ec3b-4309-a741-d772b7077399 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:13.196042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d7b96-ec3b-4309-a741-d772b7077399 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:13.257842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:13.346916 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 275/550] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:26:13.353148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a0685-0657-40fa-ba80-2a2775faffcb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:13.409960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71301356-1305-4dae-8dbb-ed1a7980ed35 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:13.410885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71301356-1305-4dae-8dbb-ed1a7980ed35 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:13.411612 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71301356-1305-4dae-8dbb-ed1a7980ed35 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:13.414894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 276/551] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:13.436633 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1978147-fd78-440e-9179-f7452172cc0e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:13.437528 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1978147-fd78-440e-9179-f7452172cc0e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:13.438414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1978147-fd78-440e-9179-f7452172cc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:13.487915 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f1978147-fd78-440e-9179-f7452172cc0e None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:13.492907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f1978147-fd78-440e-9179-f7452172cc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:13.665226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a0685-0657-40fa-ba80-2a2775faffcb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:13.680249 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 277/552] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:13.767956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443e41a-eef7-4f84-a5c3-10ff43384e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:13.768904 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443e41a-eef7-4f84-a5c3-10ff43384e86 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:13.769381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443e41a-eef7-4f84-a5c3-10ff43384e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:13.769844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443e41a-eef7-4f84-a5c3-10ff43384e86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:13.797068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443e41a-eef7-4f84-a5c3-10ff43384e86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:13.800380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443e41a-eef7-4f84-a5c3-10ff43384e86 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7443e41a-eef7-4f84-a5c3-10ff43384e86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:13.843365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:13.845154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:13.848238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:13.848238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:13.848238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:13.848238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 276/553] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 493 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:13.854122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1978147-fd78-440e-9179-f7452172cc0e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:14.116016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:14.155856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1978147-fd78-440e-9179-f7452172cc0e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:14.225075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b54871-fb93-417b-8320-75a8033b6077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:14.226107 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b54871-fb93-417b-8320-75a8033b6077 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:14.226631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b54871-fb93-417b-8320-75a8033b6077 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:14.227114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b54871-fb93-417b-8320-75a8033b6077 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:14.228286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b54871-fb93-417b-8320-75a8033b6077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:14.228828 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b54871-fb93-417b-8320-75a8033b6077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:14.231423 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b54871-fb93-417b-8320-75a8033b6077 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:14.254113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 278/554] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:14.263857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443e41a-eef7-4f84-a5c3-10ff43384e86 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:14.320303 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:14.517068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443e41a-eef7-4f84-a5c3-10ff43384e86 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:14.544930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b54871-fb93-417b-8320-75a8033b6077 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:14.609120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab7f42-4571-4ea7-84f7-78a938dd149e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:14.610835 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab7f42-4571-4ea7-84f7-78a938dd149e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:14.611712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab7f42-4571-4ea7-84f7-78a938dd149e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:14.612466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab7f42-4571-4ea7-84f7-78a938dd149e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:14.632582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab7f42-4571-4ea7-84f7-78a938dd149e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:14.635835 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab7f42-4571-4ea7-84f7-78a938dd149e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-85ab7f42-4571-4ea7-84f7-78a938dd149e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:14.680434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:14.682771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:14.684419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:14.684419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:14.684419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:14.684419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:14.684419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:14.684419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:14.684419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:14.684419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:14.689861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 277/555] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:14.694636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab7f42-4571-4ea7-84f7-78a938dd149e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:14.818552 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b54871-fb93-417b-8320-75a8033b6077 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:14.887895 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ecdb6-4977-435e-aa5c-6e849d5f5562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:14.888967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ecdb6-4977-435e-aa5c-6e849d5f5562 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:14.889374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ecdb6-4977-435e-aa5c-6e849d5f5562 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:14.889826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ecdb6-4977-435e-aa5c-6e849d5f5562 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:14.906522 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:14.907301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ecdb6-4977-435e-aa5c-6e849d5f5562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:14.908948 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5f3ecdb6-4977-435e-aa5c-6e849d5f5562 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:14.913497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:14.914834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 279/556] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:14.916301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d0b40-4e8c-4437-960d-37637901d0a5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:14.917186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d0b40-4e8c-4437-960d-37637901d0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:14.917186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d0b40-4e8c-4437-960d-37637901d0a5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:14.922806 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 280/557] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:14.925479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d0b40-4e8c-4437-960d-37637901d0a5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:14.947028 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab7f42-4571-4ea7-84f7-78a938dd149e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:14.990609 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add63c24-07fb-45d0-8ef3-74dd956a7604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:14.991223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63c24-07fb-45d0-8ef3-74dd956a7604 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:14.991498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63c24-07fb-45d0-8ef3-74dd956a7604 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:14.991764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63c24-07fb-45d0-8ef3-74dd956a7604 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:14.992458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add63c24-07fb-45d0-8ef3-74dd956a7604 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:14.992753 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add63c24-07fb-45d0-8ef3-74dd956a7604 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:14.994227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add63c24-07fb-45d0-8ef3-74dd956a7604 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:15.008277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 278/558] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:15.009624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50f9c-4f4d-4ed5-8372-27c929d92968 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:15.009908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50f9c-4f4d-4ed5-8372-27c929d92968 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:15.010173 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50f9c-4f4d-4ed5-8372-27c929d92968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:15.039876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-32f50f9c-4f4d-4ed5-8372-27c929d92968 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:15.041505 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-32f50f9c-4f4d-4ed5-8372-27c929d92968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:15.101964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d0b40-4e8c-4437-960d-37637901d0a5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:15.165445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 279/559] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:15.169209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b991a-008b-4f3b-927e-ed0dba593c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:15.170681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b991a-008b-4f3b-927e-ed0dba593c63 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:15.171452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b991a-008b-4f3b-927e-ed0dba593c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:15.172202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b991a-008b-4f3b-927e-ed0dba593c63 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:15.174323 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b991a-008b-4f3b-927e-ed0dba593c63 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:15.175181 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b991a-008b-4f3b-927e-ed0dba593c63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:15.175852 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f50f9c-4f4d-4ed5-8372-27c929d92968 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:15.178510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b991a-008b-4f3b-927e-ed0dba593c63 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:15.197589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 281/560] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:15.206199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b991a-008b-4f3b-927e-ed0dba593c63 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:15.292514 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:15.377429 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b991a-008b-4f3b-927e-ed0dba593c63 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:15.401887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:15.460168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168ea4b-3056-43b3-908b-0fe750c5d452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:15.461243 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168ea4b-3056-43b3-908b-0fe750c5d452 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:15.461734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168ea4b-3056-43b3-908b-0fe750c5d452 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:15.462254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168ea4b-3056-43b3-908b-0fe750c5d452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:15.506012 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168ea4b-3056-43b3-908b-0fe750c5d452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:15.508840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168ea4b-3056-43b3-908b-0fe750c5d452 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:15.532826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f50f9c-4f4d-4ed5-8372-27c929d92968 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2168ea4b-3056-43b3-908b-0fe750c5d452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:15.570129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:15.571143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:15.572020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:15.572020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:15.572020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:15.579772 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 282/561] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:15.588189 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168ea4b-3056-43b3-908b-0fe750c5d452 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:15.667123 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b3c1c-306a-4cc0-a9e7-950ec697d522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:15.667676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b3c1c-306a-4cc0-a9e7-950ec697d522 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:15.667956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b3c1c-306a-4cc0-a9e7-950ec697d522 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:15.668225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b3c1c-306a-4cc0-a9e7-950ec697d522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:15.668969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b3c1c-306a-4cc0-a9e7-950ec697d522 None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=e60a2198022c4f979a6e5e9057558c33) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:15.669274 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b3c1c-306a-4cc0-a9e7-950ec697d522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:15.670796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b3c1c-306a-4cc0-a9e7-950ec697d522 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:15.681539 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 280/562] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:15 2022] GET /identity/v3/role_assignments?user.id=e60a2198022c4f979a6e5e9057558c33&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 259 bytes in 515 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:15.787752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b72140-51e5-467c-8a28-b7cb5b53b30d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:15.788080 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b72140-51e5-467c-8a28-b7cb5b53b30d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:15.788302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b72140-51e5-467c-8a28-b7cb5b53b30d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:15.789688 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 281/563] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:15.804607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b72140-51e5-467c-8a28-b7cb5b53b30d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:15.824733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168ea4b-3056-43b3-908b-0fe750c5d452 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:15.836585 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:15.880102 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee91c3-4ea1-4462-b803-bc23d4e8e262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:15.880988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee91c3-4ea1-4462-b803-bc23d4e8e262 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:15.881623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee91c3-4ea1-4462-b803-bc23d4e8e262 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:15.881905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee91c3-4ea1-4462-b803-bc23d4e8e262 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:15.882616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee91c3-4ea1-4462-b803-bc23d4e8e262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:15.882920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee91c3-4ea1-4462-b803-bc23d4e8e262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:15.884358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee91c3-4ea1-4462-b803-bc23d4e8e262 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:15.900462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 283/564] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:15.901919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680174c-1931-444a-b8a0-e6c18963adc3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:15.902229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680174c-1931-444a-b8a0-e6c18963adc3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:15.902507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680174c-1931-444a-b8a0-e6c18963adc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:15.926218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b680174c-1931-444a-b8a0-e6c18963adc3 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:15.927262 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b680174c-1931-444a-b8a0-e6c18963adc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:15.969731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b72140-51e5-467c-8a28-b7cb5b53b30d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:16.009137 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:16.014272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9283cf-bda5-46cb-bccf-415bb0fcc7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:16.014925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9283cf-bda5-46cb-bccf-415bb0fcc7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:16.015209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9283cf-bda5-46cb-bccf-415bb0fcc7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:16.015478 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9283cf-bda5-46cb-bccf-415bb0fcc7e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:16.033595 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9283cf-bda5-46cb-bccf-415bb0fcc7e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:16.035111 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 284/565] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3f9283cf-bda5-46cb-bccf-415bb0fcc7e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:16.035287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:16.037161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 282/566] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:16.038437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9283cf-bda5-46cb-bccf-415bb0fcc7e6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:16.039723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506292f-a542-4a62-a4de-00c9a80e42f9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:16.039723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506292f-a542-4a62-a4de-00c9a80e42f9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:16.039961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506292f-a542-4a62-a4de-00c9a80e42f9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:16.046645 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 285/567] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:16.046645 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506292f-a542-4a62-a4de-00c9a80e42f9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:16.170771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:16.179250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506292f-a542-4a62-a4de-00c9a80e42f9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:16.207307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9283cf-bda5-46cb-bccf-415bb0fcc7e6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:16.230563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57b3c6-556f-4e9b-b6a9-2e4ace1effae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:16.231693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57b3c6-556f-4e9b-b6a9-2e4ace1effae None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:16.232177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57b3c6-556f-4e9b-b6a9-2e4ace1effae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:16.232688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57b3c6-556f-4e9b-b6a9-2e4ace1effae None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:16.233570 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57b3c6-556f-4e9b-b6a9-2e4ace1effae None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:16.234094 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57b3c6-556f-4e9b-b6a9-2e4ace1effae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:16.235736 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57b3c6-556f-4e9b-b6a9-2e4ace1effae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:16.246762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 286/568] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:16.249149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fcd7ce-5ff5-44e5-b52a-0a5414e77958 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:16.249666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fcd7ce-5ff5-44e5-b52a-0a5414e77958 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:16.250135 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fcd7ce-5ff5-44e5-b52a-0a5414e77958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:16.274833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-01fcd7ce-5ff5-44e5-b52a-0a5414e77958 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:16.276483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-01fcd7ce-5ff5-44e5-b52a-0a5414e77958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:16.285324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb91c4f-26af-4ae9-aca9-646ee0fd7add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:16.286914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb91c4f-26af-4ae9-aca9-646ee0fd7add None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:16.287210 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb91c4f-26af-4ae9-aca9-646ee0fd7add None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:16.287599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb91c4f-26af-4ae9-aca9-646ee0fd7add None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:16.296504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb91c4f-26af-4ae9-aca9-646ee0fd7add None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:16.297946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb91c4f-26af-4ae9-aca9-646ee0fd7add None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0cb91c4f-26af-4ae9-aca9-646ee0fd7add None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:16.311378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:16.313282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:16.315248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:16.315248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:16.315248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:16.315248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:16.315248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:16.315248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:16.315248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:16.315248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:16.315248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 08:26:16.315248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:16.317318 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 283/569] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:26:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:16.318320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb91c4f-26af-4ae9-aca9-646ee0fd7add None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:16.379871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:16.400160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 287/570] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:16.401349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fcd7ce-5ff5-44e5-b52a-0a5414e77958 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:16.434760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb91c4f-26af-4ae9-aca9-646ee0fd7add None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:16.485257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c313a-a501-4689-a64f-75e9eaf5f634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:16.487176 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c313a-a501-4689-a64f-75e9eaf5f634 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:16.487677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c313a-a501-4689-a64f-75e9eaf5f634 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:16.488533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c313a-a501-4689-a64f-75e9eaf5f634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:16.513509 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c313a-a501-4689-a64f-75e9eaf5f634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:16.515752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c313a-a501-4689-a64f-75e9eaf5f634 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:16.552442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-336c313a-a501-4689-a64f-75e9eaf5f634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:16.557895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:16.559956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:16.561765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:16.561765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:16.561765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:16.561765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 284/571] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:16.561765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c313a-a501-4689-a64f-75e9eaf5f634 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:16.588401 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fcd7ce-5ff5-44e5-b52a-0a5414e77958 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:16.681832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecab11-52bf-4aec-a7c5-1c48397e0732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:16.683911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecab11-52bf-4aec-a7c5-1c48397e0732 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:16.684673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecab11-52bf-4aec-a7c5-1c48397e0732 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:16.685191 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecab11-52bf-4aec-a7c5-1c48397e0732 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:16.687442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecab11-52bf-4aec-a7c5-1c48397e0732 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:16.688186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecab11-52bf-4aec-a7c5-1c48397e0732 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:16.691240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecab11-52bf-4aec-a7c5-1c48397e0732 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:16.693822 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c313a-a501-4689-a64f-75e9eaf5f634 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:16.698569 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:16.727920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 288/572] 10.222.0.44 () {56 vars in 1137 bytes} [Sun Sep 11 08:26:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:16.729225 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecab11-52bf-4aec-a7c5-1c48397e0732 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:16.749422 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785645b-ee03-4186-b927-5aaa6360ea3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:16.750125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785645b-ee03-4186-b927-5aaa6360ea3a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:16.750559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785645b-ee03-4186-b927-5aaa6360ea3a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:16.750947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785645b-ee03-4186-b927-5aaa6360ea3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:16.770691 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785645b-ee03-4186-b927-5aaa6360ea3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3785645b-ee03-4186-b927-5aaa6360ea3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:16.781256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:16.784226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:16.787430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:16.787430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:16.787430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:16.787430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:16.787430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:16.787430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:16.787430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:16.787430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:16.787430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:16.787430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:16.787430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:16.787430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:16.787430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 285/573] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:16.792523 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785645b-ee03-4186-b927-5aaa6360ea3a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:16.856546 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecab11-52bf-4aec-a7c5-1c48397e0732 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:16.899548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4c20a-3743-4502-9717-191b5dc2debc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:16.900220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4c20a-3743-4502-9717-191b5dc2debc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:16.900499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4c20a-3743-4502-9717-191b5dc2debc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:16.901072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4c20a-3743-4502-9717-191b5dc2debc None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:16.901423 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4c20a-3743-4502-9717-191b5dc2debc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:16.901731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4c20a-3743-4502-9717-191b5dc2debc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:16.903208 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4c20a-3743-4502-9717-191b5dc2debc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:16.920874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 289/574] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:16.921877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4c20a-3743-4502-9717-191b5dc2debc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:16.934654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:16.957061 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785645b-ee03-4186-b927-5aaa6360ea3a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:16.997156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc9b5b-8b22-4d51-8cf5-3c0395a98871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:16.997938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc9b5b-8b22-4d51-8cf5-3c0395a98871 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:16.998236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc9b5b-8b22-4d51-8cf5-3c0395a98871 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:16.998493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc9b5b-8b22-4d51-8cf5-3c0395a98871 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:17.011487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc9b5b-8b22-4d51-8cf5-3c0395a98871 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e8fc9b5b-8b22-4d51-8cf5-3c0395a98871 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:17.017107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:17.020224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 286/575] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:17.021650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc9b5b-8b22-4d51-8cf5-3c0395a98871 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:17.052210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4c20a-3743-4502-9717-191b5dc2debc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:17.061352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:17.096211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222fb86-c93a-4bce-b1b1-7489cfa91447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:17.096863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222fb86-c93a-4bce-b1b1-7489cfa91447 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:17.097167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222fb86-c93a-4bce-b1b1-7489cfa91447 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:17.097434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222fb86-c93a-4bce-b1b1-7489cfa91447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:17.111040 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222fb86-c93a-4bce-b1b1-7489cfa91447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:17.112893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222fb86-c93a-4bce-b1b1-7489cfa91447 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9222fb86-c93a-4bce-b1b1-7489cfa91447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:17.135313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:17.136494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:17.137677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:17.137677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:17.137677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:17.141275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 290/576] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:17.142301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222fb86-c93a-4bce-b1b1-7489cfa91447 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:17.160890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc9b5b-8b22-4d51-8cf5-3c0395a98871 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:17.204362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ffcfe-360b-4406-8101-9372c0e337d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:17.205826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ffcfe-360b-4406-8101-9372c0e337d2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:17.206282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ffcfe-360b-4406-8101-9372c0e337d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:17.206721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ffcfe-360b-4406-8101-9372c0e337d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:17.207621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ffcfe-360b-4406-8101-9372c0e337d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:17.208079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ffcfe-360b-4406-8101-9372c0e337d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:17.209655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ffcfe-360b-4406-8101-9372c0e337d2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:17.219879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:17.221230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 287/577] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:17.222111 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ffcfe-360b-4406-8101-9372c0e337d2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:17.278418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222fb86-c93a-4bce-b1b1-7489cfa91447 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:17.319473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e5b43-0329-4279-8756-b38c37d70ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:17.320119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e5b43-0329-4279-8756-b38c37d70ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:17.320396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e5b43-0329-4279-8756-b38c37d70ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:17.320670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e5b43-0329-4279-8756-b38c37d70ea5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:17.329308 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e5b43-0329-4279-8756-b38c37d70ea5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:17.330877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e5b43-0329-4279-8756-b38c37d70ea5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c28e5b43-0329-4279-8756-b38c37d70ea5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:17.344391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:17.345993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:17.348201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:17.348201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:17.348201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:17.348201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:17.348201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:17.348201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:17.348201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:17.348201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:17.348201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 08:26:17.348201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:17.350572 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 291/578] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:26:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:17.351592 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e5b43-0329-4279-8756-b38c37d70ea5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:17.363308 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:17.381029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ffcfe-360b-4406-8101-9372c0e337d2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:17.421574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b2af-e6f6-4f47-965f-43a717e74384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:17.422379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b2af-e6f6-4f47-965f-43a717e74384 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:17.422779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b2af-e6f6-4f47-965f-43a717e74384 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:17.423051 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b2af-e6f6-4f47-965f-43a717e74384 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:17.424271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987b2af-e6f6-4f47-965f-43a717e74384 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:17.424569 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987b2af-e6f6-4f47-965f-43a717e74384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:17.425980 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987b2af-e6f6-4f47-965f-43a717e74384 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:17.436086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 288/579] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:17.441807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b2af-e6f6-4f47-965f-43a717e74384 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:17.552629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b2af-e6f6-4f47-965f-43a717e74384 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:17.556288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e5b43-0329-4279-8756-b38c37d70ea5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:17.588251 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:17.596922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88739da-71c6-4534-ace5-8c8e095aabdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:17.597999 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88739da-71c6-4534-ace5-8c8e095aabdc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:17.605377 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88739da-71c6-4534-ace5-8c8e095aabdc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:17.605377 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88739da-71c6-4534-ace5-8c8e095aabdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:17.612872 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c5700-4ced-43d5-b952-46328a9e486f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:17.613502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c5700-4ced-43d5-b952-46328a9e486f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:17.613779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c5700-4ced-43d5-b952-46328a9e486f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:17.614045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c5700-4ced-43d5-b952-46328a9e486f None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:17.614734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c5700-4ced-43d5-b952-46328a9e486f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:17.615030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c5700-4ced-43d5-b952-46328a9e486f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:17.616404 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c5700-4ced-43d5-b952-46328a9e486f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:17.619817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88739da-71c6-4534-ace5-8c8e095aabdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:17.622544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88739da-71c6-4534-ace5-8c8e095aabdc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:17.637143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:17.646286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 292/580] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:17.646286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c5700-4ced-43d5-b952-46328a9e486f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e88739da-71c6-4534-ace5-8c8e095aabdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:17.650604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:17.651640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:17.652796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:17.652796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:17.652796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:17.652796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 289/581] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:17.654032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88739da-71c6-4534-ace5-8c8e095aabdc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:17.808630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c5700-4ced-43d5-b952-46328a9e486f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:17.822982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88739da-71c6-4534-ace5-8c8e095aabdc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:17.875435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aea766-bd3e-4985-a24a-e7c49b5c1cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:17.876122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aea766-bd3e-4985-a24a-e7c49b5c1cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:17.876423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aea766-bd3e-4985-a24a-e7c49b5c1cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:17.876705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aea766-bd3e-4985-a24a-e7c49b5c1cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:17.883505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cd500-a9d2-4fc8-91d9-11840624c59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:17.884557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cd500-a9d2-4fc8-91d9-11840624c59c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:17.885032 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cd500-a9d2-4fc8-91d9-11840624c59c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:17.887677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cd500-a9d2-4fc8-91d9-11840624c59c None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:17.889788 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cd500-a9d2-4fc8-91d9-11840624c59c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:17.890312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cd500-a9d2-4fc8-91d9-11840624c59c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:17.890599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aea766-bd3e-4985-a24a-e7c49b5c1cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:17.892100 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aea766-bd3e-4985-a24a-e7c49b5c1cb7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:17.894549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6cd500-a9d2-4fc8-91d9-11840624c59c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-38aea766-bd3e-4985-a24a-e7c49b5c1cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:17.915030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:17.916877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:17.918722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:17.918722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:17.918722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:17.918722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 290/582] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:17.919174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aea766-bd3e-4985-a24a-e7c49b5c1cb7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:17.925930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 293/583] 10.222.0.44 () {56 vars in 1137 bytes} [Sun Sep 11 08:26:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:17.926944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cd500-a9d2-4fc8-91d9-11840624c59c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:17.976062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:18.019527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:18.080721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aea766-bd3e-4985-a24a-e7c49b5c1cb7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:18.091470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cd500-a9d2-4fc8-91d9-11840624c59c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:18.136588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afa4c3-2f38-42b1-a64a-41072347e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:18.136884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c57b7-e25e-41aa-9798-d3d035cfbd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:18.137319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c57b7-e25e-41aa-9798-d3d035cfbd22 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:18.137599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c57b7-e25e-41aa-9798-d3d035cfbd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:18.137755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afa4c3-2f38-42b1-a64a-41072347e2dd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:18.137890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c57b7-e25e-41aa-9798-d3d035cfbd22 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:18.138234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afa4c3-2f38-42b1-a64a-41072347e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:18.138702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c57b7-e25e-41aa-9798-d3d035cfbd22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:18.138863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afa4c3-2f38-42b1-a64a-41072347e2dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:18.138997 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c57b7-e25e-41aa-9798-d3d035cfbd22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:18.140577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c57b7-e25e-41aa-9798-d3d035cfbd22 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:18.153975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afa4c3-2f38-42b1-a64a-41072347e2dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:18.156886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afa4c3-2f38-42b1-a64a-41072347e2dd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:18.160649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 294/584] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:18.161526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c57b7-e25e-41aa-9798-d3d035cfbd22 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b3afa4c3-2f38-42b1-a64a-41072347e2dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:18.200233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:18.202436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:18.203882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:18.203882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:18.203882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:18.203882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:18.203882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:18.203882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:18.203882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:18.203882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:18.203882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 291/585] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:18.204440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afa4c3-2f38-42b1-a64a-41072347e2dd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:18.300103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:18.304435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c57b7-e25e-41aa-9798-d3d035cfbd22 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:18.341261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afa4c3-2f38-42b1-a64a-41072347e2dd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:18.353128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101f9db-f536-4075-8a4d-e79aef5ada52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:18.354066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101f9db-f536-4075-8a4d-e79aef5ada52 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:18.354543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101f9db-f536-4075-8a4d-e79aef5ada52 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:18.354918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101f9db-f536-4075-8a4d-e79aef5ada52 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:18.355960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101f9db-f536-4075-8a4d-e79aef5ada52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:18.356455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101f9db-f536-4075-8a4d-e79aef5ada52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:18.358729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101f9db-f536-4075-8a4d-e79aef5ada52 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:18.361325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:18.378480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 295/586] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:18.380382 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1496d2-f853-4809-ba74-f31c7572965b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:18.380815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1496d2-f853-4809-ba74-f31c7572965b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:18.381264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1496d2-f853-4809-ba74-f31c7572965b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:18.383517 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 296/587] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:18.386289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1496d2-f853-4809-ba74-f31c7572965b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:18.397111 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3579b6-d5e6-4900-b5c4-5d462d6acdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:18.397837 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3579b6-d5e6-4900-b5c4-5d462d6acdef None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:18.398199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3579b6-d5e6-4900-b5c4-5d462d6acdef None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:18.398537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3579b6-d5e6-4900-b5c4-5d462d6acdef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:18.411493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3579b6-d5e6-4900-b5c4-5d462d6acdef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:18.413191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3579b6-d5e6-4900-b5c4-5d462d6acdef None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-bf3579b6-d5e6-4900-b5c4-5d462d6acdef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:18.429690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:18.431651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:18.433774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:18.433774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:18.433774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:18.433774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 292/588] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:18.433774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3579b6-d5e6-4900-b5c4-5d462d6acdef None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:18.544837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1496d2-f853-4809-ba74-f31c7572965b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:18.584056 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3579b6-d5e6-4900-b5c4-5d462d6acdef None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:18.587355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62acda3-c55e-4412-8531-1e8f21e50fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:18.588250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62acda3-c55e-4412-8531-1e8f21e50fee None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:18.588566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62acda3-c55e-4412-8531-1e8f21e50fee None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:18.588869 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62acda3-c55e-4412-8531-1e8f21e50fee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:18.598924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62acda3-c55e-4412-8531-1e8f21e50fee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:18.600835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62acda3-c55e-4412-8531-1e8f21e50fee None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d62acda3-c55e-4412-8531-1e8f21e50fee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:18.616126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:18.617998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:18.620673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:18.620673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:18.620673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:18.620673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:18.620673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:18.620673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:18.620673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:18.620673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:18.620673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:26:18.620673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:18.620673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 297/589] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 08:26:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:18.620673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62acda3-c55e-4412-8531-1e8f21e50fee None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:18.631928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22849f4-5b84-42ab-9950-931eb9b1eab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:18.632543 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22849f4-5b84-42ab-9950-931eb9b1eab1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:18.632806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22849f4-5b84-42ab-9950-931eb9b1eab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:18.633062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22849f4-5b84-42ab-9950-931eb9b1eab1 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:18.633758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22849f4-5b84-42ab-9950-931eb9b1eab1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:18.634057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22849f4-5b84-42ab-9950-931eb9b1eab1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:18.635560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22849f4-5b84-42ab-9950-931eb9b1eab1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:18.647678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:18.653865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 293/590] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:18.653865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13727fa6-ca45-4b08-8ac0-c61cc642712c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:18.653865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13727fa6-ca45-4b08-8ac0-c61cc642712c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:18.653865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13727fa6-ca45-4b08-8ac0-c61cc642712c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:18.669279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-13727fa6-ca45-4b08-8ac0-c61cc642712c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:18.670530 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-13727fa6-ca45-4b08-8ac0-c61cc642712c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:18.693670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:18.758132 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62acda3-c55e-4412-8531-1e8f21e50fee None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:18.760858 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 294/591] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:18.762334 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13727fa6-ca45-4b08-8ac0-c61cc642712c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:18.817185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02586d09-b8f6-4a7b-9a24-137d11b6020a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:18.818263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02586d09-b8f6-4a7b-9a24-137d11b6020a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:18.818753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02586d09-b8f6-4a7b-9a24-137d11b6020a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:18.819257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02586d09-b8f6-4a7b-9a24-137d11b6020a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:18.836305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02586d09-b8f6-4a7b-9a24-137d11b6020a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:18.839122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02586d09-b8f6-4a7b-9a24-137d11b6020a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-02586d09-b8f6-4a7b-9a24-137d11b6020a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:18.859882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:18.861502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:18.862623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:18.862623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:18.862623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:18.862623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:18.862623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:18.862623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:18.862623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:18.862623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:18.862623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 08:26:18.862623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:18.870103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 298/592] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:18.871695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02586d09-b8f6-4a7b-9a24-137d11b6020a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:18.929461 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13727fa6-ca45-4b08-8ac0-c61cc642712c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:18.970055 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8799f-7099-40f4-aacc-9a03a301b466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:18.970772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8799f-7099-40f4-aacc-9a03a301b466 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:18.971390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8799f-7099-40f4-aacc-9a03a301b466 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:18.971390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8799f-7099-40f4-aacc-9a03a301b466 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:18.971913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8799f-7099-40f4-aacc-9a03a301b466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:18.972197 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8799f-7099-40f4-aacc-9a03a301b466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:18.973614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8799f-7099-40f4-aacc-9a03a301b466 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:18.993498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 295/593] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:18.993498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8799f-7099-40f4-aacc-9a03a301b466 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:19.008298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:19.020469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:19.042953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02586d09-b8f6-4a7b-9a24-137d11b6020a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:19.086682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79462a1-8be5-4698-9048-979312b88540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:19.087624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79462a1-8be5-4698-9048-979312b88540 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:19.088065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79462a1-8be5-4698-9048-979312b88540 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:19.088991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79462a1-8be5-4698-9048-979312b88540 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:19.089454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79462a1-8be5-4698-9048-979312b88540 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:19.090232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79462a1-8be5-4698-9048-979312b88540 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:19.092692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79462a1-8be5-4698-9048-979312b88540 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:19.114013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 299/594] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 08:26:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:19.115140 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79462a1-8be5-4698-9048-979312b88540 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:19.128435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8799f-7099-40f4-aacc-9a03a301b466 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:19.164415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f868d8c-3f9c-4635-ae46-fad2a8dbf9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:19.165078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f868d8c-3f9c-4635-ae46-fad2a8dbf9d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:19.165346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f868d8c-3f9c-4635-ae46-fad2a8dbf9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:19.165613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f868d8c-3f9c-4635-ae46-fad2a8dbf9d5 None admin] PATH_INFO: `/v3/projects/811863e9c94d44a39647800f306505f0/users/434c9e87bda343659d2f410a6cdbf805/roles/a76c253615fd43a09d6dc924dbe16553` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:19.182589 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f868d8c-3f9c-4635-ae46-fad2a8dbf9d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=811863e9c94d44a39647800f306505f0, user_id=434c9e87bda343659d2f410a6cdbf805, role_id=a76c253615fd43a09d6dc924dbe16553)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:19.184049 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f868d8c-3f9c-4635-ae46-fad2a8dbf9d5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:19.226293 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79462a1-8be5-4698-9048-979312b88540 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:19.254484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 296/595] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:26:18 2022] PUT /identity/v3/projects/811863e9c94d44a39647800f306505f0/users/434c9e87bda343659d2f410a6cdbf805/roles/a76c253615fd43a09d6dc924dbe16553 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:26:19.255917 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f868d8c-3f9c-4635-ae46-fad2a8dbf9d5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:19.276882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd4af1-658a-4450-9170-0b93b51a8852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:19.277999 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd4af1-658a-4450-9170-0b93b51a8852 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:19.278563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd4af1-658a-4450-9170-0b93b51a8852 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:19.279096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd4af1-658a-4450-9170-0b93b51a8852 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:19.280579 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd4af1-658a-4450-9170-0b93b51a8852 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:19.281213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd4af1-658a-4450-9170-0b93b51a8852 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:19.284509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd4af1-658a-4450-9170-0b93b51a8852 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:19.289642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:19.311926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 300/596] 10.222.0.44 () {56 vars in 1139 bytes} [Sun Sep 11 08:26:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:19.313535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd4af1-658a-4450-9170-0b93b51a8852 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:19.379545 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:19.446753 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f868d8c-3f9c-4635-ae46-fad2a8dbf9d5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:19.480771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd4af1-658a-4450-9170-0b93b51a8852 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:19.495299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbeb6d9-a9d4-4034-b056-d763bc1c0e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:19.496061 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbeb6d9-a9d4-4034-b056-d763bc1c0e39 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:19.496550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbeb6d9-a9d4-4034-b056-d763bc1c0e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:19.496872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbeb6d9-a9d4-4034-b056-d763bc1c0e39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:19.506408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbeb6d9-a9d4-4034-b056-d763bc1c0e39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:19.507987 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbeb6d9-a9d4-4034-b056-d763bc1c0e39 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1fbeb6d9-a9d4-4034-b056-d763bc1c0e39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:19.521641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:19.524638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:19.526819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:19.526819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:19.526819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:19.526819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:19.526819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:19.526819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:19.526819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:19.526819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:19.526819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 297/597] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:19.526819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbeb6d9-a9d4-4034-b056-d763bc1c0e39 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:19.553059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ba244-65c4-4c5f-8185-2aaf44cd2d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:19.554028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ba244-65c4-4c5f-8185-2aaf44cd2d6d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:19.554540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ba244-65c4-4c5f-8185-2aaf44cd2d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:19.554820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ba244-65c4-4c5f-8185-2aaf44cd2d6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:19.568754 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ba244-65c4-4c5f-8185-2aaf44cd2d6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-cf3ba244-65c4-4c5f-8185-2aaf44cd2d6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:19.573682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:19.575246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 301/598] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:19.576938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ba244-65c4-4c5f-8185-2aaf44cd2d6d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:19.651874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:19.666894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbeb6d9-a9d4-4034-b056-d763bc1c0e39 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:19.693559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:19.706930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d62762-61a8-46da-a020-8f70600348db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:19.707576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d62762-61a8-46da-a020-8f70600348db None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:19.707841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d62762-61a8-46da-a020-8f70600348db None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:19.708105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d62762-61a8-46da-a020-8f70600348db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:19.722006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d62762-61a8-46da-a020-8f70600348db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:19.723379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ba244-65c4-4c5f-8185-2aaf44cd2d6d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:19.723573 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d62762-61a8-46da-a020-8f70600348db None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b3d62762-61a8-46da-a020-8f70600348db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:19.739938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:19.741856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:19.743678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:19.743678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:19.743678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:19.743678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 298/599] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:19.743678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d62762-61a8-46da-a020-8f70600348db None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:19.784437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d61300-63f7-405f-acfa-4fab01e1c379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:19.785068 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d61300-63f7-405f-acfa-4fab01e1c379 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:19.785355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d61300-63f7-405f-acfa-4fab01e1c379 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:19.785611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d61300-63f7-405f-acfa-4fab01e1c379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:19.801367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d61300-63f7-405f-acfa-4fab01e1c379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:19.804609 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d61300-63f7-405f-acfa-4fab01e1c379 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-a3d61300-63f7-405f-acfa-4fab01e1c379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:19.834018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:19.835237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:19.836318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:19.836318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:19.836318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:19.843143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 302/600] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:19.843143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d61300-63f7-405f-acfa-4fab01e1c379 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:19.891614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d62762-61a8-46da-a020-8f70600348db None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:19.932226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc047ef2-ffa1-4ed4-a554-15dbb35c2af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:19.933236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc047ef2-ffa1-4ed4-a554-15dbb35c2af9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:19.933236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc047ef2-ffa1-4ed4-a554-15dbb35c2af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:19.933527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc047ef2-ffa1-4ed4-a554-15dbb35c2af9 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:19.933977 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc047ef2-ffa1-4ed4-a554-15dbb35c2af9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:19.934387 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc047ef2-ffa1-4ed4-a554-15dbb35c2af9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:19.935830 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc047ef2-ffa1-4ed4-a554-15dbb35c2af9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:19.950238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 299/601] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:19.951106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc047ef2-ffa1-4ed4-a554-15dbb35c2af9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:19.979462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d61300-63f7-405f-acfa-4fab01e1c379 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:19.989681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:19.993839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:20.036275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61158dc-f77c-4b4a-8982-b0950c9fee24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:20.036940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61158dc-f77c-4b4a-8982-b0950c9fee24 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:20.037179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61158dc-f77c-4b4a-8982-b0950c9fee24 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:20.037449 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61158dc-f77c-4b4a-8982-b0950c9fee24 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:20.038170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61158dc-f77c-4b4a-8982-b0950c9fee24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:20.038482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61158dc-f77c-4b4a-8982-b0950c9fee24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:20.040032 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61158dc-f77c-4b4a-8982-b0950c9fee24 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:20.049846 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 303/602] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:20.055239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61158dc-f77c-4b4a-8982-b0950c9fee24 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:20.090952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc047ef2-ffa1-4ed4-a554-15dbb35c2af9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:20.142975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e6a6a-3e7d-4943-81b2-0aecb0bfead7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:20.143620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e6a6a-3e7d-4943-81b2-0aecb0bfead7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:20.143893 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e6a6a-3e7d-4943-81b2-0aecb0bfead7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:20.144159 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e6a6a-3e7d-4943-81b2-0aecb0bfead7 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:20.144984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e6a6a-3e7d-4943-81b2-0aecb0bfead7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:20.145157 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e6a6a-3e7d-4943-81b2-0aecb0bfead7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:20.146606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e6a6a-3e7d-4943-81b2-0aecb0bfead7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:20.161072 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 300/603] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:20.161936 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e6a6a-3e7d-4943-81b2-0aecb0bfead7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:20.200367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61158dc-f77c-4b4a-8982-b0950c9fee24 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:20.249449 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0b5b8-0657-46ac-95ba-5debae46f1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:20.250563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b5b8-0657-46ac-95ba-5debae46f1db None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:20.251061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b5b8-0657-46ac-95ba-5debae46f1db None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:20.251581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b5b8-0657-46ac-95ba-5debae46f1db None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:20.252859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0b5b8-0657-46ac-95ba-5debae46f1db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:20.253296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0b5b8-0657-46ac-95ba-5debae46f1db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:20.254904 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0b5b8-0657-46ac-95ba-5debae46f1db None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:20.271804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 304/604] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:20.272670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0b5b8-0657-46ac-95ba-5debae46f1db None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:20.284360 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:20.291892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e6a6a-3e7d-4943-81b2-0aecb0bfead7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:20.317095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:20.332887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf35f60-5949-4515-8e89-bfa13993a1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:20.335713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf35f60-5949-4515-8e89-bfa13993a1c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:20.335713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf35f60-5949-4515-8e89-bfa13993a1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:20.335713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf35f60-5949-4515-8e89-bfa13993a1c8 None admin] PATH_INFO: `/v3/projects/811863e9c94d44a39647800f306505f0/users/7b86978df61245ae88fead154193fcd4/roles/a76c253615fd43a09d6dc924dbe16553` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:20.353111 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf35f60-5949-4515-8e89-bfa13993a1c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=811863e9c94d44a39647800f306505f0, user_id=7b86978df61245ae88fead154193fcd4, role_id=a76c253615fd43a09d6dc924dbe16553)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:20.354768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf35f60-5949-4515-8e89-bfa13993a1c8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:20.408652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0b5b8-0657-46ac-95ba-5debae46f1db None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:20.414354 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 301/605] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:26:20 2022] PUT /identity/v3/projects/811863e9c94d44a39647800f306505f0/users/7b86978df61245ae88fead154193fcd4/roles/a76c253615fd43a09d6dc924dbe16553 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:26:20.415466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf35f60-5949-4515-8e89-bfa13993a1c8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:20.472841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a4e95-77cc-4d9e-a823-22503421d064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:20.474541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a4e95-77cc-4d9e-a823-22503421d064 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:20.475029 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a4e95-77cc-4d9e-a823-22503421d064 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:20.475445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a4e95-77cc-4d9e-a823-22503421d064 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:20.492187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a4e95-77cc-4d9e-a823-22503421d064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:20.494716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a4e95-77cc-4d9e-a823-22503421d064 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-de4a4e95-77cc-4d9e-a823-22503421d064 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:20.515351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:20.517027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:20.518183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:20.518183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:20.518183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:20.520303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 305/606] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:20.521546 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a4e95-77cc-4d9e-a823-22503421d064 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:20.566320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf35f60-5949-4515-8e89-bfa13993a1c8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:20.630419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82227e95-e69a-4e9f-a682-723eae0edb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:20.631319 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82227e95-e69a-4e9f-a682-723eae0edb6d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:20.631838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82227e95-e69a-4e9f-a682-723eae0edb6d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:20.632745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82227e95-e69a-4e9f-a682-723eae0edb6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:20.638666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:20.645222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82227e95-e69a-4e9f-a682-723eae0edb6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:20.648156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82227e95-e69a-4e9f-a682-723eae0edb6d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:20.662560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a4e95-77cc-4d9e-a823-22503421d064 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-82227e95-e69a-4e9f-a682-723eae0edb6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:20.670598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:20.671703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:20.672755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:20.672755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:20.672755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:20.672755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:20.672755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:20.672755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:20.672755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:20.672755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:20.682827 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 302/607] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:20.684448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82227e95-e69a-4e9f-a682-723eae0edb6d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:20.700443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:20.719714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2bd15-c8ad-4745-8418-21b7e5488e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:20.720395 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2bd15-c8ad-4745-8418-21b7e5488e96 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:20.720667 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2bd15-c8ad-4745-8418-21b7e5488e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:20.720935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2bd15-c8ad-4745-8418-21b7e5488e96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:20.728973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2bd15-c8ad-4745-8418-21b7e5488e96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:20.730098 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2bd15-c8ad-4745-8418-21b7e5488e96 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-eee2bd15-c8ad-4745-8418-21b7e5488e96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:20.745239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:20.746980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:20.748745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:20.748745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:20.748745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:20.748745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:20.748745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:20.748745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:20.748745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:20.748745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:20.748745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 306/608] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:20.748745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2bd15-c8ad-4745-8418-21b7e5488e96 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:20.868752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2bd15-c8ad-4745-8418-21b7e5488e96 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:20.909915 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f235bec-e410-4fec-b912-fb62ac996e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:20.910564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f235bec-e410-4fec-b912-fb62ac996e63 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:20.910843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f235bec-e410-4fec-b912-fb62ac996e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:20.911121 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f235bec-e410-4fec-b912-fb62ac996e63 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:20.911805 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f235bec-e410-4fec-b912-fb62ac996e63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:20.912099 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f235bec-e410-4fec-b912-fb62ac996e63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:20.913689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f235bec-e410-4fec-b912-fb62ac996e63 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:20.920351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82227e95-e69a-4e9f-a682-723eae0edb6d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:20.924536 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:20.933784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 307/609] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:20.933784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f235bec-e410-4fec-b912-fb62ac996e63 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:21.004529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb49e14-d383-424f-a3b1-6dbb684a2d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:21.006823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb49e14-d383-424f-a3b1-6dbb684a2d4a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:21.007790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb49e14-d383-424f-a3b1-6dbb684a2d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:21.008591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb49e14-d383-424f-a3b1-6dbb684a2d4a None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:21.010436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb49e14-d383-424f-a3b1-6dbb684a2d4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:21.011317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb49e14-d383-424f-a3b1-6dbb684a2d4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:21.014630 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb49e14-d383-424f-a3b1-6dbb684a2d4a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:21.046457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 303/610] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:21.049145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb49e14-d383-424f-a3b1-6dbb684a2d4a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:21.060602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f235bec-e410-4fec-b912-fb62ac996e63 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:21.102652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791d2e9-9d4d-41d7-96cf-e60f693d8a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:21.103300 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791d2e9-9d4d-41d7-96cf-e60f693d8a93 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:21.103593 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791d2e9-9d4d-41d7-96cf-e60f693d8a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:21.103867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791d2e9-9d4d-41d7-96cf-e60f693d8a93 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:21.104574 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791d2e9-9d4d-41d7-96cf-e60f693d8a93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:21.104883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791d2e9-9d4d-41d7-96cf-e60f693d8a93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:21.106469 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791d2e9-9d4d-41d7-96cf-e60f693d8a93 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:21.122244 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 308/611] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:21.123556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791d2e9-9d4d-41d7-96cf-e60f693d8a93 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:21.194998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:21.205602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:21.262482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791d2e9-9d4d-41d7-96cf-e60f693d8a93 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:21.263824 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb49e14-d383-424f-a3b1-6dbb684a2d4a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:21.317508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903a84a-70ad-48df-845b-da876f9cdb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:21.318207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903a84a-70ad-48df-845b-da876f9cdb06 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:21.318501 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903a84a-70ad-48df-845b-da876f9cdb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:21.318739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903a84a-70ad-48df-845b-da876f9cdb06 None admin] PATH_INFO: `/v3/projects/811863e9c94d44a39647800f306505f0/users/58c8502b91ea4ba79e85818727422410/roles/a76c253615fd43a09d6dc924dbe16553` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:21.338858 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903a84a-70ad-48df-845b-da876f9cdb06 None admin] RBAC: Authorizing `identity:create_grant(project_id=811863e9c94d44a39647800f306505f0, user_id=58c8502b91ea4ba79e85818727422410, role_id=a76c253615fd43a09d6dc924dbe16553)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:21.340856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903a84a-70ad-48df-845b-da876f9cdb06 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:21.342806 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85994ee3-76f3-4dd1-9179-0cbd5c5b630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:21.344527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85994ee3-76f3-4dd1-9179-0cbd5c5b630b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:21.346447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85994ee3-76f3-4dd1-9179-0cbd5c5b630b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:21.347058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85994ee3-76f3-4dd1-9179-0cbd5c5b630b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:21.362031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85994ee3-76f3-4dd1-9179-0cbd5c5b630b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:21.364738 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85994ee3-76f3-4dd1-9179-0cbd5c5b630b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-85994ee3-76f3-4dd1-9179-0cbd5c5b630b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:21.386940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:21.388070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:21.389133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:21.389133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:21.389133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:21.389133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:21.389133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:21.389133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:21.389133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:21.389133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:21.389133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 08:26:21.389133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:21.396832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 309/612] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:26:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:21.396832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85994ee3-76f3-4dd1-9179-0cbd5c5b630b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:21.473110 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 304/613] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:26:21 2022] PUT /identity/v3/projects/811863e9c94d44a39647800f306505f0/users/58c8502b91ea4ba79e85818727422410/roles/a76c253615fd43a09d6dc924dbe16553 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:26:21.475279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903a84a-70ad-48df-845b-da876f9cdb06 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:21.622949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85994ee3-76f3-4dd1-9179-0cbd5c5b630b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:21.669019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:21.702385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606f599-4cdf-48cd-bbc9-e76b725c4de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:21.704153 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606f599-4cdf-48cd-bbc9-e76b725c4de2 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:21.705553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606f599-4cdf-48cd-bbc9-e76b725c4de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:21.706370 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606f599-4cdf-48cd-bbc9-e76b725c4de2 None admin] PATH_INFO: `/v3/projects/811863e9c94d44a39647800f306505f0/users/92867e15dcf649adad78bf7a29a996f9/roles/a76c253615fd43a09d6dc924dbe16553` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:21.740343 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606f599-4cdf-48cd-bbc9-e76b725c4de2 None admin] RBAC: Authorizing `identity:create_grant(project_id=811863e9c94d44a39647800f306505f0, user_id=92867e15dcf649adad78bf7a29a996f9, role_id=a76c253615fd43a09d6dc924dbe16553)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:21.743771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606f599-4cdf-48cd-bbc9-e76b725c4de2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:21.812679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903a84a-70ad-48df-845b-da876f9cdb06 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:21.863483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 310/614] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:26:21 2022] PUT /identity/v3/projects/811863e9c94d44a39647800f306505f0/users/92867e15dcf649adad78bf7a29a996f9/roles/a76c253615fd43a09d6dc924dbe16553 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:26:21.864717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:21.903279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2581b8-640f-4e2b-ac04-183f4a792aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:21.904932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2581b8-640f-4e2b-ac04-183f4a792aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:21.905772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2581b8-640f-4e2b-ac04-183f4a792aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:21.906586 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2581b8-640f-4e2b-ac04-183f4a792aa5 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:21.909173 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2581b8-640f-4e2b-ac04-183f4a792aa5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:21.910108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2581b8-640f-4e2b-ac04-183f4a792aa5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:21.913204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2581b8-640f-4e2b-ac04-183f4a792aa5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:21.955144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 305/615] 10.222.0.44 () {56 vars in 1135 bytes} [Sun Sep 11 08:26:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 479 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:21.964529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606f599-4cdf-48cd-bbc9-e76b725c4de2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:22.168096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606f599-4cdf-48cd-bbc9-e76b725c4de2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:22.211688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0027e-b240-47b4-bb54-7633ef47d60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:22.212337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0027e-b240-47b4-bb54-7633ef47d60d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:22.212740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0027e-b240-47b4-bb54-7633ef47d60d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:22.212883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0027e-b240-47b4-bb54-7633ef47d60d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:22.226564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0027e-b240-47b4-bb54-7633ef47d60d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:22.228019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0027e-b240-47b4-bb54-7633ef47d60d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:22.243701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770b4b9-9ac8-401d-afc3-848578984b6f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:22.244287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770b4b9-9ac8-401d-afc3-848578984b6f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:22.244287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770b4b9-9ac8-401d-afc3-848578984b6f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-91f0027e-b240-47b4-bb54-7633ef47d60d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:22.245228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:22.247463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:22.250699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:22.250699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:22.250699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:22.253574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 306/616] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:22.253836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 311/617] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:22.255685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2004fd-2578-4b83-9064-bb0fb1fc311c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:22.255969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2004fd-2578-4b83-9064-bb0fb1fc311c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:22.256244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2004fd-2578-4b83-9064-bb0fb1fc311c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:22.261621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0027e-b240-47b4-bb54-7633ef47d60d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:22.293689 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1d2004fd-2578-4b83-9064-bb0fb1fc311c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:22.295456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1d2004fd-2578-4b83-9064-bb0fb1fc311c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:22.318203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:22.402239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 307/618] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:22.470388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0027e-b240-47b4-bb54-7633ef47d60d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:22.508197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6cc83-1baf-468a-91f3-cab3a81e0795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:22.509047 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6cc83-1baf-468a-91f3-cab3a81e0795 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:22.509425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6cc83-1baf-468a-91f3-cab3a81e0795 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:22.509798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6cc83-1baf-468a-91f3-cab3a81e0795 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:22.510589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6cc83-1baf-468a-91f3-cab3a81e0795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:22.511002 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6cc83-1baf-468a-91f3-cab3a81e0795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:22.512513 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6cc83-1baf-468a-91f3-cab3a81e0795 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:22.526722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 312/619] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:22.535672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2004fd-2578-4b83-9064-bb0fb1fc311c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:22.672660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2004fd-2578-4b83-9064-bb0fb1fc311c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:22.696698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:22.723369 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0be3fb-ae57-43b6-bb22-a24e82d40ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:22.723934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0be3fb-ae57-43b6-bb22-a24e82d40ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:22.724212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0be3fb-ae57-43b6-bb22-a24e82d40ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:22.724486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0be3fb-ae57-43b6-bb22-a24e82d40ce9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:22.743390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0be3fb-ae57-43b6-bb22-a24e82d40ce9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:22.749172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0be3fb-ae57-43b6-bb22-a24e82d40ce9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-df0be3fb-ae57-43b6-bb22-a24e82d40ce9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:22.762476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:22.763783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:22.764956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:22.764956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:22.764956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:22.764956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:22.764956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:22.764956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:22.764956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:22.764956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:22.764956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 308/620] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:22.775783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6cc83-1baf-468a-91f3-cab3a81e0795 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:22.879189 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:22.900747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6cc83-1baf-468a-91f3-cab3a81e0795 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:22.922905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5148ae-6804-4cd7-afdd-1c5cecb9dc31 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:22.924014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5148ae-6804-4cd7-afdd-1c5cecb9dc31 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:22.924301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5148ae-6804-4cd7-afdd-1c5cecb9dc31 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:22.925900 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 309/621] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:22.932895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f90b8-f081-47d6-8338-a33b9aa42c6c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:22.933745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f90b8-f081-47d6-8338-a33b9aa42c6c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:22.934437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f90b8-f081-47d6-8338-a33b9aa42c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:22.948907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc375833-3272-4cf3-9aa2-284b74fbe448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:22.949536 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc375833-3272-4cf3-9aa2-284b74fbe448 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:22.949818 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc375833-3272-4cf3-9aa2-284b74fbe448 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:22.950095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc375833-3272-4cf3-9aa2-284b74fbe448 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:22.950844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc375833-3272-4cf3-9aa2-284b74fbe448 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:22.951157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc375833-3272-4cf3-9aa2-284b74fbe448 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:22.952842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc375833-3272-4cf3-9aa2-284b74fbe448 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:22.972046 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 313/622] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:22.981876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc375833-3272-4cf3-9aa2-284b74fbe448 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:22.988528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-137f90b8-f081-47d6-8338-a33b9aa42c6c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:22.989815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-137f90b8-f081-47d6-8338-a33b9aa42c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:23.124477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 310/623] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:23.125437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f90b8-f081-47d6-8338-a33b9aa42c6c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:23.183088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc375833-3272-4cf3-9aa2-284b74fbe448 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:23.242625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:23.251242 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f90b8-f081-47d6-8338-a33b9aa42c6c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:23.259131 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d224a-0870-4c5a-9a40-a40edb90c6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:23.259789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d224a-0870-4c5a-9a40-a40edb90c6ce None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:23.260412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d224a-0870-4c5a-9a40-a40edb90c6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:23.260412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d224a-0870-4c5a-9a40-a40edb90c6ce None admin] PATH_INFO: `/v3/projects/811863e9c94d44a39647800f306505f0/users/e60a2198022c4f979a6e5e9057558c33/roles/a76c253615fd43a09d6dc924dbe16553` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:23.292764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:23.300067 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d224a-0870-4c5a-9a40-a40edb90c6ce None admin] RBAC: Authorizing `identity:create_grant(project_id=811863e9c94d44a39647800f306505f0, user_id=e60a2198022c4f979a6e5e9057558c33, role_id=a76c253615fd43a09d6dc924dbe16553)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:23.301593 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d224a-0870-4c5a-9a40-a40edb90c6ce None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:23.322125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1c583-2e5f-45fe-92a3-04a28ad21a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:23.323831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1c583-2e5f-45fe-92a3-04a28ad21a55 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:23.324092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1c583-2e5f-45fe-92a3-04a28ad21a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:23.324316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1c583-2e5f-45fe-92a3-04a28ad21a55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:23.337750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1c583-2e5f-45fe-92a3-04a28ad21a55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-eae1c583-2e5f-45fe-92a3-04a28ad21a55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:23.343128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:23.344083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:23.345774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:23.345774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:23.345774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:23.345774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:23.345774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:23.345774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:23.345774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:23.345774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:23.345774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:23.345774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:23.345774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:23.345774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:23.345774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 311/624] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:23.353798 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1c583-2e5f-45fe-92a3-04a28ad21a55 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:23.387784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 314/625] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:26:22 2022] PUT /identity/v3/projects/811863e9c94d44a39647800f306505f0/users/e60a2198022c4f979a6e5e9057558c33/roles/a76c253615fd43a09d6dc924dbe16553 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:26:23.482359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1c583-2e5f-45fe-92a3-04a28ad21a55 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:23.521877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ba0af-6a14-44af-925a-30d1e09af63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:23.522519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ba0af-6a14-44af-925a-30d1e09af63a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:23.523026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ba0af-6a14-44af-925a-30d1e09af63a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:23.523360 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ba0af-6a14-44af-925a-30d1e09af63a None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:23.524112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ba0af-6a14-44af-925a-30d1e09af63a None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:23.524483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ba0af-6a14-44af-925a-30d1e09af63a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:23.525929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ba0af-6a14-44af-925a-30d1e09af63a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:23.536640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 312/626] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:23.545081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d224a-0870-4c5a-9a40-a40edb90c6ce None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:23.693188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ba0af-6a14-44af-925a-30d1e09af63a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:23.713033 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d224a-0870-4c5a-9a40-a40edb90c6ce None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:23.718404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:23.790819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881349e-bce0-4bdc-be74-92a15bc28bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:23.791926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881349e-bce0-4bdc-be74-92a15bc28bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:23.792423 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881349e-bce0-4bdc-be74-92a15bc28bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:23.792906 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881349e-bce0-4bdc-be74-92a15bc28bd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:23.821316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ba0af-6a14-44af-925a-30d1e09af63a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:23.827094 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881349e-bce0-4bdc-be74-92a15bc28bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:23.837581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881349e-bce0-4bdc-be74-92a15bc28bd4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:23.859140 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac8120-5660-4da9-875b-2e01543d78bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:23.859838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac8120-5660-4da9-875b-2e01543d78bd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:23.860120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac8120-5660-4da9-875b-2e01543d78bd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:23.860392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac8120-5660-4da9-875b-2e01543d78bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:23.872986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac8120-5660-4da9-875b-2e01543d78bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e9ac8120-5660-4da9-875b-2e01543d78bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:23.880651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:23.881829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:23.882930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:23.882930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:23.882930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:23.882930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:23.882930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:23.882930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:23.882930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:23.882930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:23.882930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:23.882930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:23.882930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:23.882930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:23.882930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 313/627] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:23.891294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac8120-5660-4da9-875b-2e01543d78bd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3881349e-bce0-4bdc-be74-92a15bc28bd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:23.891693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:23.893470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:23.894479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:23.894479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:23.894479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:23.894479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 315/628] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:23.903221 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881349e-bce0-4bdc-be74-92a15bc28bd4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:24.059309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:24.077933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac8120-5660-4da9-875b-2e01543d78bd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:24.081780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:24.123619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3599e0-3fa8-486e-923b-832bb618359b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:24.124233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3599e0-3fa8-486e-923b-832bb618359b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:24.124567 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3599e0-3fa8-486e-923b-832bb618359b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:24.124836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3599e0-3fa8-486e-923b-832bb618359b None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:24.125555 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3599e0-3fa8-486e-923b-832bb618359b None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:24.125861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3599e0-3fa8-486e-923b-832bb618359b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:24.134191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3599e0-3fa8-486e-923b-832bb618359b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:24.137758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 314/629] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:24.147440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3599e0-3fa8-486e-923b-832bb618359b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:24.227001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881349e-bce0-4bdc-be74-92a15bc28bd4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:24.266187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3599e0-3fa8-486e-923b-832bb618359b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:24.321338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e355e-5458-417b-8faf-3845a2addf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:24.323052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e355e-5458-417b-8faf-3845a2addf6d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:24.323394 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e355e-5458-417b-8faf-3845a2addf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:24.323632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e355e-5458-417b-8faf-3845a2addf6d None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:24.324279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e355e-5458-417b-8faf-3845a2addf6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:24.324585 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e355e-5458-417b-8faf-3845a2addf6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:24.326052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e355e-5458-417b-8faf-3845a2addf6d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:24.343093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7692-7bf5-4fc5-849d-cca1f931e4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:24.343758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7692-7bf5-4fc5-849d-cca1f931e4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:24.344035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7692-7bf5-4fc5-849d-cca1f931e4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:24.344304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7692-7bf5-4fc5-849d-cca1f931e4e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:24.359932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 316/630] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:24.362794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:24.367364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b7692-7bf5-4fc5-849d-cca1f931e4e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:24.369799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e355e-5458-417b-8faf-3845a2addf6d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:24.379045 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b7692-7bf5-4fc5-849d-cca1f931e4e5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-fc6b7692-7bf5-4fc5-849d-cca1f931e4e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:24.405147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:24.406247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:24.407169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:24.407169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:24.407169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:24.411947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 315/631] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:24.421059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7692-7bf5-4fc5-849d-cca1f931e4e5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:24.552431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e355e-5458-417b-8faf-3845a2addf6d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:24.556020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:24.622300 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d6909-0f3e-48b9-9458-191ddfde909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:24.622864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d6909-0f3e-48b9-9458-191ddfde909d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:24.623136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d6909-0f3e-48b9-9458-191ddfde909d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:24.623399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d6909-0f3e-48b9-9458-191ddfde909d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:24.632452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d6909-0f3e-48b9-9458-191ddfde909d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:24.633923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d6909-0f3e-48b9-9458-191ddfde909d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:24.645264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7692-7bf5-4fc5-849d-cca1f931e4e5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-949d6909-0f3e-48b9-9458-191ddfde909d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:24.646939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:24.647912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:24.648848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:24.648848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:24.648848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:24.648848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:24.648848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:24.648848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:24.648848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:24.648848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:24.648848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 08:26:24.648848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:24.648848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 317/632] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:26:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:24.659206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d6909-0f3e-48b9-9458-191ddfde909d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:24.713388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77770c7-701c-4752-a5a5-91d73b588ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:24.714401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77770c7-701c-4752-a5a5-91d73b588ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:24.714889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77770c7-701c-4752-a5a5-91d73b588ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:24.715626 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77770c7-701c-4752-a5a5-91d73b588ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:24.716558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77770c7-701c-4752-a5a5-91d73b588ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:24.717066 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77770c7-701c-4752-a5a5-91d73b588ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:24.719702 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77770c7-701c-4752-a5a5-91d73b588ec9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:24.746208 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 316/633] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:24.752648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77770c7-701c-4752-a5a5-91d73b588ec9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:24.832275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:24.897125 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d6909-0f3e-48b9-9458-191ddfde909d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:24.899381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77770c7-701c-4752-a5a5-91d73b588ec9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:24.953212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198e3ef-9e06-402a-a446-d5df7d40e341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:24.955539 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198e3ef-9e06-402a-a446-d5df7d40e341 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:24.956483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198e3ef-9e06-402a-a446-d5df7d40e341 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:24.957128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198e3ef-9e06-402a-a446-d5df7d40e341 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:24.975317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198e3ef-9e06-402a-a446-d5df7d40e341 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:24.978837 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198e3ef-9e06-402a-a446-d5df7d40e341 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-8198e3ef-9e06-402a-a446-d5df7d40e341 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:25.000469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:25.001731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:25.002974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:25.002974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:25.002974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:25.002974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:25.002974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:25.002974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:25.002974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:25.002974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:25.002974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 08:26:25.002974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:25.002974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 317/634] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:26:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:25.004707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82942c-6dcb-45dd-93c1-0c8e9d32dabd None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:25.004707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82942c-6dcb-45dd-93c1-0c8e9d32dabd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:25.004707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82942c-6dcb-45dd-93c1-0c8e9d32dabd None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:25.009657 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 318/635] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:25.010918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82942c-6dcb-45dd-93c1-0c8e9d32dabd None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:25.012219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c68eb-4d95-4cb3-ab11-30a766ec7a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:25.013508 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c68eb-4d95-4cb3-ab11-30a766ec7a56 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:25.014045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c68eb-4d95-4cb3-ab11-30a766ec7a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:25.014700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c68eb-4d95-4cb3-ab11-30a766ec7a56 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:25.015699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c68eb-4d95-4cb3-ab11-30a766ec7a56 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:25.016268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c68eb-4d95-4cb3-ab11-30a766ec7a56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:25.019020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c68eb-4d95-4cb3-ab11-30a766ec7a56 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:25.040967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 318/636] 10.222.0.44 () {56 vars in 1137 bytes} [Sun Sep 11 08:26:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:25.042446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127e564-5beb-4199-9155-9da2eb220796 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:25.042805 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127e564-5beb-4199-9155-9da2eb220796 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:25.043156 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127e564-5beb-4199-9155-9da2eb220796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:25.062520 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9127e564-5beb-4199-9155-9da2eb220796 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:25.064371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9127e564-5beb-4199-9155-9da2eb220796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:25.083574 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:25.146387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 319/637] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:25.147446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127e564-5beb-4199-9155-9da2eb220796 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:25.190682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82942c-6dcb-45dd-93c1-0c8e9d32dabd None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:25.207320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:25.254521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467676ba-db6f-451e-8f92-f00d3847279c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:25.255165 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467676ba-db6f-451e-8f92-f00d3847279c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:25.255433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467676ba-db6f-451e-8f92-f00d3847279c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:25.255707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467676ba-db6f-451e-8f92-f00d3847279c None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:25.256575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467676ba-db6f-451e-8f92-f00d3847279c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:25.256925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467676ba-db6f-451e-8f92-f00d3847279c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:25.258585 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467676ba-db6f-451e-8f92-f00d3847279c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:25.292655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 319/638] 10.222.0.44 () {56 vars in 1137 bytes} [Sun Sep 11 08:26:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:25.295126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efc11d-51c3-40d9-bd1d-715b9f10582c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:25.295600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efc11d-51c3-40d9-bd1d-715b9f10582c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:25.296122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efc11d-51c3-40d9-bd1d-715b9f10582c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:25.302237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 320/639] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:25.302237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efc11d-51c3-40d9-bd1d-715b9f10582c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:25.363288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127e564-5beb-4199-9155-9da2eb220796 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:25.435732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5518ec3-b25f-4a54-bcb4-7913ce67dcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:25.436838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5518ec3-b25f-4a54-bcb4-7913ce67dcbf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:25.437400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5518ec3-b25f-4a54-bcb4-7913ce67dcbf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:25.437942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5518ec3-b25f-4a54-bcb4-7913ce67dcbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:25.486664 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:25.513886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efc11d-51c3-40d9-bd1d-715b9f10582c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:25.557613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5518ec3-b25f-4a54-bcb4-7913ce67dcbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:25.559408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5518ec3-b25f-4a54-bcb4-7913ce67dcbf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:25.573748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f00f5-7094-46cb-98f9-7bab8cab1261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:25.574770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f00f5-7094-46cb-98f9-7bab8cab1261 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:25.575240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f00f5-7094-46cb-98f9-7bab8cab1261 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:25.575697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f00f5-7094-46cb-98f9-7bab8cab1261 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:25.641224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:25.701376 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f00f5-7094-46cb-98f9-7bab8cab1261 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:25.704103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f00f5-7094-46cb-98f9-7bab8cab1261 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:25.722675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 320/640] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:26:25.722675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c87ffb-4361-48f4-a22d-7d7a5255f5b3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:25.722675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c87ffb-4361-48f4-a22d-7d7a5255f5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:25.722675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c87ffb-4361-48f4-a22d-7d7a5255f5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:25.737873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-55c87ffb-4361-48f4-a22d-7d7a5255f5b3 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:25.739133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-55c87ffb-4361-48f4-a22d-7d7a5255f5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:25.796511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:25.828283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 321/641] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:25.828283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c87ffb-4361-48f4-a22d-7d7a5255f5b3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:25.889835 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 321/642] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:26:25.890868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f00f5-7094-46cb-98f9-7bab8cab1261 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:25.974164 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c87ffb-4361-48f4-a22d-7d7a5255f5b3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:25.976299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:26.020328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f00f5-7094-46cb-98f9-7bab8cab1261 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:26.025353 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f279bcd-2d90-4815-a30f-a03bb7f34b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:26.026452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f279bcd-2d90-4815-a30f-a03bb7f34b13 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:26.027074 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f279bcd-2d90-4815-a30f-a03bb7f34b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:26.027540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f279bcd-2d90-4815-a30f-a03bb7f34b13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:26.042697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f279bcd-2d90-4815-a30f-a03bb7f34b13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2f279bcd-2d90-4815-a30f-a03bb7f34b13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:26.050336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:26.051553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:26.052388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:26.052388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:26.052388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:26.052388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:26.052388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:26.052388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:26.052388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:26.052388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:26.052388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:26.052388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:26.052388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:26.052388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:26.055708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 322/643] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:26.056971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f279bcd-2d90-4815-a30f-a03bb7f34b13 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:26.059195 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a0078-cb7b-4b8f-b520-24e7c6caaf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:26.059850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a0078-cb7b-4b8f-b520-24e7c6caaf1d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:26.060128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a0078-cb7b-4b8f-b520-24e7c6caaf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:26.060604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a0078-cb7b-4b8f-b520-24e7c6caaf1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:26.074240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a0078-cb7b-4b8f-b520-24e7c6caaf1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:26.076184 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a0078-cb7b-4b8f-b520-24e7c6caaf1d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-990a0078-cb7b-4b8f-b520-24e7c6caaf1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:26.098306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:26.099394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:26.100307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:26.100307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:26.100307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:26.102608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 322/644] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:26.103794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a0078-cb7b-4b8f-b520-24e7c6caaf1d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:26.144108 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:26.228890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f279bcd-2d90-4815-a30f-a03bb7f34b13 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:26.234540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a0078-cb7b-4b8f-b520-24e7c6caaf1d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:26.262841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:26.281227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9d1a0-db6b-47b8-8a96-2e7b1be5ac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:26.281544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a9911-b515-48dc-954d-47437dcc5262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:26.281910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9911-b515-48dc-954d-47437dcc5262 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:26.282229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9911-b515-48dc-954d-47437dcc5262 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:26.282410 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9d1a0-db6b-47b8-8a96-2e7b1be5ac4a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:26.282586 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9911-b515-48dc-954d-47437dcc5262 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:26.282875 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9d1a0-db6b-47b8-8a96-2e7b1be5ac4a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:26.283282 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9d1a0-db6b-47b8-8a96-2e7b1be5ac4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:26.283450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a9911-b515-48dc-954d-47437dcc5262 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:26.283602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a9911-b515-48dc-954d-47437dcc5262 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:26.285019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a9911-b515-48dc-954d-47437dcc5262 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:26.295969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9d1a0-db6b-47b8-8a96-2e7b1be5ac4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:26.302190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9d1a0-db6b-47b8-8a96-2e7b1be5ac4a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:26.303591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 323/645] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:26.304840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a9911-b515-48dc-954d-47437dcc5262 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8ff9d1a0-db6b-47b8-8a96-2e7b1be5ac4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:26.316698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:26.318571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:26.320540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:26.320540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:26.320540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:26.320540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 323/646] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:26.320540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9d1a0-db6b-47b8-8a96-2e7b1be5ac4a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:26.457527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a9911-b515-48dc-954d-47437dcc5262 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:26.469809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:26.477349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9d1a0-db6b-47b8-8a96-2e7b1be5ac4a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:26.509082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648f825-c0dc-45b6-8e90-1409c5231d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:26.509626 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648f825-c0dc-45b6-8e90-1409c5231d14 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:26.509895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648f825-c0dc-45b6-8e90-1409c5231d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:26.510200 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648f825-c0dc-45b6-8e90-1409c5231d14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:26.519219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad69ef-18d5-4172-8aac-49e5fa8aa8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:26.519835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad69ef-18d5-4172-8aac-49e5fa8aa8fa None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:26.520172 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad69ef-18d5-4172-8aac-49e5fa8aa8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:26.520448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad69ef-18d5-4172-8aac-49e5fa8aa8fa None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:26.521095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad69ef-18d5-4172-8aac-49e5fa8aa8fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:26.521393 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad69ef-18d5-4172-8aac-49e5fa8aa8fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:26.522822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad69ef-18d5-4172-8aac-49e5fa8aa8fa None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:26.526747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648f825-c0dc-45b6-8e90-1409c5231d14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c648f825-c0dc-45b6-8e90-1409c5231d14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:26.531848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:26.533284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:26.534474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:26.534474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:26.534474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:26.534474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:26.534474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:26.534474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:26.534474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:26.534474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:26.534474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:26.534474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:26.534474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:26.534474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:26.534474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 324/647] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:26.535772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648f825-c0dc-45b6-8e90-1409c5231d14 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:26.545864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 324/648] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:26.545864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad69ef-18d5-4172-8aac-49e5fa8aa8fa None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:26.597217 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:26.680851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648f825-c0dc-45b6-8e90-1409c5231d14 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:26.685483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad69ef-18d5-4172-8aac-49e5fa8aa8fa None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:26.720921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdab827-e0e2-47b4-8070-92a2de15ee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:26.721554 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdab827-e0e2-47b4-8070-92a2de15ee34 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:26.721891 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdab827-e0e2-47b4-8070-92a2de15ee34 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:26.722279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdab827-e0e2-47b4-8070-92a2de15ee34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:26.728759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71791bd7-042e-4b60-9707-62555c2a40c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:26.729794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71791bd7-042e-4b60-9707-62555c2a40c8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:26.730315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71791bd7-042e-4b60-9707-62555c2a40c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:26.730800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71791bd7-042e-4b60-9707-62555c2a40c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:26.731962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71791bd7-042e-4b60-9707-62555c2a40c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:26.732497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71791bd7-042e-4b60-9707-62555c2a40c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:26.734445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdab827-e0e2-47b4-8070-92a2de15ee34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:26.735137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71791bd7-042e-4b60-9707-62555c2a40c8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:26.736613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdab827-e0e2-47b4-8070-92a2de15ee34 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2bdab827-e0e2-47b4-8070-92a2de15ee34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:26.753432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:26.755026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:26.756780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:26.756780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:26.756780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:26.756780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 325/649] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:26.756780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5ce95-a554-4fb5-a5c9-b240e9df84d1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:26.756780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5ce95-a554-4fb5-a5c9-b240e9df84d1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:26.756780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5ce95-a554-4fb5-a5c9-b240e9df84d1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:26.758349 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 326/650] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:26.758595 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5ce95-a554-4fb5-a5c9-b240e9df84d1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:26.761771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 325/651] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:26.762566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71791bd7-042e-4b60-9707-62555c2a40c8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:26.783913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:26.873861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:26.883446 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5ce95-a554-4fb5-a5c9-b240e9df84d1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:26.897511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71791bd7-042e-4b60-9707-62555c2a40c8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:26.922777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73619f-5e33-4266-90d2-b37d1bc4ae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:26.923939 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73619f-5e33-4266-90d2-b37d1bc4ae60 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:26.924210 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73619f-5e33-4266-90d2-b37d1bc4ae60 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:26.924478 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73619f-5e33-4266-90d2-b37d1bc4ae60 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:26.925135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73619f-5e33-4266-90d2-b37d1bc4ae60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:26.925427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73619f-5e33-4266-90d2-b37d1bc4ae60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:26.926841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73619f-5e33-4266-90d2-b37d1bc4ae60 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:26.935045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8af97-267f-43b9-8548-6f05b662b727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:26.935684 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8af97-267f-43b9-8548-6f05b662b727 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:26.935963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8af97-267f-43b9-8548-6f05b662b727 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:26.936218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8af97-267f-43b9-8548-6f05b662b727 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:26.938141 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 327/652] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:26.939075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73619f-5e33-4266-90d2-b37d1bc4ae60 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:26.950249 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8af97-267f-43b9-8548-6f05b662b727 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:26.952007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8af97-267f-43b9-8548-6f05b662b727 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-dcb8af97-267f-43b9-8548-6f05b662b727 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:26.965721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:26.967648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:26.968827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:26.968827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:26.968827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:26.968827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:26.968827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:26.968827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:26.968827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:26.968827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:26.968827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 326/653] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:26.968827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4252e08-ae74-4901-b38d-6e6b4ff31606 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:26.968827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4252e08-ae74-4901-b38d-6e6b4ff31606 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:26.969779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4252e08-ae74-4901-b38d-6e6b4ff31606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:26.991271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e4252e08-ae74-4901-b38d-6e6b4ff31606 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:26.992497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4252e08-ae74-4901-b38d-6e6b4ff31606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:27.060536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73619f-5e33-4266-90d2-b37d1bc4ae60 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:27.090373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 327/654] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:27.090373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4252e08-ae74-4901-b38d-6e6b4ff31606 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:27.090373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:27.114458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff439b-07f7-4adc-a9a9-6c83593b35fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:27.115506 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff439b-07f7-4adc-a9a9-6c83593b35fe None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:27.116738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff439b-07f7-4adc-a9a9-6c83593b35fe None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:27.117419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff439b-07f7-4adc-a9a9-6c83593b35fe None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:27.118748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff439b-07f7-4adc-a9a9-6c83593b35fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:27.119119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff439b-07f7-4adc-a9a9-6c83593b35fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:27.121745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff439b-07f7-4adc-a9a9-6c83593b35fe None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:27.143331 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 328/655] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:27.144649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff439b-07f7-4adc-a9a9-6c83593b35fe None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:27.166466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:27.233049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4252e08-ae74-4901-b38d-6e6b4ff31606 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:27.272749 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d276b-179e-473b-b198-071bfea7c724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:27.273383 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d276b-179e-473b-b198-071bfea7c724 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:27.273651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d276b-179e-473b-b198-071bfea7c724 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:27.273922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d276b-179e-473b-b198-071bfea7c724 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:27.281732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d276b-179e-473b-b198-071bfea7c724 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:27.283815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d276b-179e-473b-b198-071bfea7c724 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:27.294401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff439b-07f7-4adc-a9a9-6c83593b35fe None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ef0d276b-179e-473b-b198-071bfea7c724 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:27.296149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:27.297867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:27.300018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:27.300018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:27.300018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:27.300018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:27.300018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:27.300018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:27.300018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:27.300018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:27.300018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 328/656] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:27.300018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d276b-179e-473b-b198-071bfea7c724 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:27.367578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425b41f-6441-41d8-9a73-2d20ef74af51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:27.368490 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425b41f-6441-41d8-9a73-2d20ef74af51 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:27.368939 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425b41f-6441-41d8-9a73-2d20ef74af51 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:27.369380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425b41f-6441-41d8-9a73-2d20ef74af51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:27.386358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425b41f-6441-41d8-9a73-2d20ef74af51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:27.387372 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:27.388417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425b41f-6441-41d8-9a73-2d20ef74af51 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9425b41f-6441-41d8-9a73-2d20ef74af51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:27.414126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:27.415356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:27.416898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:27.416898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:27.416898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:27.416898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 329/657] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:27.418491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425b41f-6441-41d8-9a73-2d20ef74af51 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:27.445333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d276b-179e-473b-b198-071bfea7c724 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:27.485323 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058613d-0d85-450c-be56-f6acbccfd459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:27.486482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058613d-0d85-450c-be56-f6acbccfd459 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:27.486929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058613d-0d85-450c-be56-f6acbccfd459 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:27.487440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058613d-0d85-450c-be56-f6acbccfd459 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:27.488502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058613d-0d85-450c-be56-f6acbccfd459 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:27.488976 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058613d-0d85-450c-be56-f6acbccfd459 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:27.490651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058613d-0d85-450c-be56-f6acbccfd459 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:27.505981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 329/658] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:27.507575 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058613d-0d85-450c-be56-f6acbccfd459 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:27.513360 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:27.557836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425b41f-6441-41d8-9a73-2d20ef74af51 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:27.592542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10250a-edc0-4d21-b92f-462480394e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:27.593162 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10250a-edc0-4d21-b92f-462480394e90 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:27.593436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10250a-edc0-4d21-b92f-462480394e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:27.593703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10250a-edc0-4d21-b92f-462480394e90 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:27.602375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10250a-edc0-4d21-b92f-462480394e90 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:27.603842 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10250a-edc0-4d21-b92f-462480394e90 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1d10250a-edc0-4d21-b92f-462480394e90 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:27.616240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:27.617484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:27.618800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:27.618800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:27.618800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:27.618800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:27.618800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:27.618800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:27.618800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:27.618800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:27.618800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:26:27.618800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:27.618800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 330/659] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 08:26:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:27.618800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10250a-edc0-4d21-b92f-462480394e90 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:27.657972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058613d-0d85-450c-be56-f6acbccfd459 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:27.698269 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:27.715973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452c834-4712-41df-9c73-c089e7ad8555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:27.717130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452c834-4712-41df-9c73-c089e7ad8555 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:27.717625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452c834-4712-41df-9c73-c089e7ad8555 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:27.718104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452c834-4712-41df-9c73-c089e7ad8555 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:27.719337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452c834-4712-41df-9c73-c089e7ad8555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:27.719877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452c834-4712-41df-9c73-c089e7ad8555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:27.722198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452c834-4712-41df-9c73-c089e7ad8555 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:27.727780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10250a-edc0-4d21-b92f-462480394e90 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:27.737222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 330/660] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:27.737835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452c834-4712-41df-9c73-c089e7ad8555 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:27.765398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:27.774301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451680b-2a24-4a7b-8e5e-5baab9d9baf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:27.776172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451680b-2a24-4a7b-8e5e-5baab9d9baf7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:27.776172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451680b-2a24-4a7b-8e5e-5baab9d9baf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:27.776172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451680b-2a24-4a7b-8e5e-5baab9d9baf7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:27.786522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451680b-2a24-4a7b-8e5e-5baab9d9baf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5451680b-2a24-4a7b-8e5e-5baab9d9baf7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:27.791358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:27.792800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 331/661] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:27.793882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451680b-2a24-4a7b-8e5e-5baab9d9baf7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:27.870771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452c834-4712-41df-9c73-c089e7ad8555 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:27.905100 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451680b-2a24-4a7b-8e5e-5baab9d9baf7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:27.919187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1491e9-2b8b-4bdc-8ec1-c97e921fa6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:27.919980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1491e9-2b8b-4bdc-8ec1-c97e921fa6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:27.920659 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1491e9-2b8b-4bdc-8ec1-c97e921fa6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:27.920937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1491e9-2b8b-4bdc-8ec1-c97e921fa6b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:27.921583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1491e9-2b8b-4bdc-8ec1-c97e921fa6b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:27.921893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1491e9-2b8b-4bdc-8ec1-c97e921fa6b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:27.923845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1491e9-2b8b-4bdc-8ec1-c97e921fa6b3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:27.936103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 331/662] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:27.936976 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1491e9-2b8b-4bdc-8ec1-c97e921fa6b3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:27.943630 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be48a2-2c26-4852-a735-a3541f4b0f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:27.944263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be48a2-2c26-4852-a735-a3541f4b0f1e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:27.944741 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be48a2-2c26-4852-a735-a3541f4b0f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:27.945006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be48a2-2c26-4852-a735-a3541f4b0f1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:27.945726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be48a2-2c26-4852-a735-a3541f4b0f1e None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=434c9e87bda343659d2f410a6cdbf805) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:27.946032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be48a2-2c26-4852-a735-a3541f4b0f1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:27.947538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be48a2-2c26-4852-a735-a3541f4b0f1e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:27.953671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 332/663] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:27 2022] GET /identity/v3/role_assignments?user.id=434c9e87bda343659d2f410a6cdbf805&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:27.954554 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be48a2-2c26-4852-a735-a3541f4b0f1e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:28.003213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:28.027067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:28.081971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1491e9-2b8b-4bdc-8ec1-c97e921fa6b3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:28.089464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be48a2-2c26-4852-a735-a3541f4b0f1e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:28.143685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226e706-bb5d-4c80-b03e-1fb2f79c2744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:28.143894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61134607-020f-46e4-a0b7-97eeb287851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:28.144243 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226e706-bb5d-4c80-b03e-1fb2f79c2744 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:28.144522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226e706-bb5d-4c80-b03e-1fb2f79c2744 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:28.144740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61134607-020f-46e4-a0b7-97eeb287851c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:28.144838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226e706-bb5d-4c80-b03e-1fb2f79c2744 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:28.145188 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61134607-020f-46e4-a0b7-97eeb287851c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:28.145554 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226e706-bb5d-4c80-b03e-1fb2f79c2744 None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=7b86978df61245ae88fead154193fcd4) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:28.145666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61134607-020f-46e4-a0b7-97eeb287851c None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:28.145868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226e706-bb5d-4c80-b03e-1fb2f79c2744 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:28.146939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61134607-020f-46e4-a0b7-97eeb287851c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:28.147440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226e706-bb5d-4c80-b03e-1fb2f79c2744 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:28.147550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61134607-020f-46e4-a0b7-97eeb287851c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:28.150104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61134607-020f-46e4-a0b7-97eeb287851c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:28.155228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 333/664] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:27 2022] GET /identity/v3/role_assignments?user.id=7b86978df61245ae88fead154193fcd4&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:28.156189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226e706-bb5d-4c80-b03e-1fb2f79c2744 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:28.194722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 332/665] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 08:26:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:28.196008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61134607-020f-46e4-a0b7-97eeb287851c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:28.331585 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226e706-bb5d-4c80-b03e-1fb2f79c2744 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:28.365682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:28.365946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61134607-020f-46e4-a0b7-97eeb287851c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:28.369171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:28.405130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5672f-3a49-47d8-a9a4-bd7d1204e036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:28.407381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5672f-3a49-47d8-a9a4-bd7d1204e036 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:28.409061 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5672f-3a49-47d8-a9a4-bd7d1204e036 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:28.409902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5672f-3a49-47d8-a9a4-bd7d1204e036 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:28.410085 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da5a51-c4fe-42ef-aa8c-5e4229a12278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:28.410710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da5a51-c4fe-42ef-aa8c-5e4229a12278 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:28.410993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da5a51-c4fe-42ef-aa8c-5e4229a12278 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:28.411270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da5a51-c4fe-42ef-aa8c-5e4229a12278 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:28.412666 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5672f-3a49-47d8-a9a4-bd7d1204e036 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:28.413389 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5672f-3a49-47d8-a9a4-bd7d1204e036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:28.416228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5672f-3a49-47d8-a9a4-bd7d1204e036 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:28.419838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da5a51-c4fe-42ef-aa8c-5e4229a12278 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:28.421343 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da5a51-c4fe-42ef-aa8c-5e4229a12278 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:28.433825 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 334/666] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:28.435323 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5672f-3a49-47d8-a9a4-bd7d1204e036 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-90da5a51-c4fe-42ef-aa8c-5e4229a12278 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:28.436632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:28.437935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:28.439959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:28.439959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:28.439959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:28.439959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:28.439959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:28.439959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:28.439959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:28.439959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:28.439959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 08:26:28.439959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:28.443740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 333/667] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:28.443740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da5a51-c4fe-42ef-aa8c-5e4229a12278 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:28.613302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5672f-3a49-47d8-a9a4-bd7d1204e036 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:28.638950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da5a51-c4fe-42ef-aa8c-5e4229a12278 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:28.676922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf0ec2-dd16-4d60-98e4-a6d840d50173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:28.677957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf0ec2-dd16-4d60-98e4-a6d840d50173 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:28.678466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf0ec2-dd16-4d60-98e4-a6d840d50173 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:28.678934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf0ec2-dd16-4d60-98e4-a6d840d50173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:28.692623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e3d29-9e70-45b6-bb4c-9ffb10cd08cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:28.693283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e3d29-9e70-45b6-bb4c-9ffb10cd08cc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:28.693578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e3d29-9e70-45b6-bb4c-9ffb10cd08cc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:28.693826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e3d29-9e70-45b6-bb4c-9ffb10cd08cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:28.710090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e3d29-9e70-45b6-bb4c-9ffb10cd08cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:28.712531 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e3d29-9e70-45b6-bb4c-9ffb10cd08cc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ec1e3d29-9e70-45b6-bb4c-9ffb10cd08cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:28.734006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:28.736415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:28.738512 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:28.738512 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:28.738512 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:28.746183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 334/668] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:28.746183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e3d29-9e70-45b6-bb4c-9ffb10cd08cc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:28.768844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:28.783271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:28.827229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf0ec2-dd16-4d60-98e4-a6d840d50173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:28.831269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf0ec2-dd16-4d60-98e4-a6d840d50173 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:28.890394 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e3d29-9e70-45b6-bb4c-9ffb10cd08cc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:28.937390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5e72f-8381-4ca8-8530-d05633541d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:28.937908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5e72f-8381-4ca8-8530-d05633541d94 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:28.938228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5e72f-8381-4ca8-8530-d05633541d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:28.938506 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5e72f-8381-4ca8-8530-d05633541d94 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:28.939818 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5e72f-8381-4ca8-8530-d05633541d94 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:28.940137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5e72f-8381-4ca8-8530-d05633541d94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:28.941583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5e72f-8381-4ca8-8530-d05633541d94 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:28.966361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 335/669] 10.222.0.44 () {56 vars in 1139 bytes} [Sun Sep 11 08:26:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:28.967326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5e72f-8381-4ca8-8530-d05633541d94 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:29.035975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 335/670] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:26:29.038251 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf0ec2-dd16-4d60-98e4-a6d840d50173 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:29.064203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:29.103746 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5e72f-8381-4ca8-8530-d05633541d94 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:29.146516 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b8521-5ec0-4726-8fcf-3037bfbf2be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:29.147148 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b8521-5ec0-4726-8fcf-3037bfbf2be7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:29.147429 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b8521-5ec0-4726-8fcf-3037bfbf2be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:29.147685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b8521-5ec0-4726-8fcf-3037bfbf2be7 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:29.148344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b8521-5ec0-4726-8fcf-3037bfbf2be7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:29.148648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b8521-5ec0-4726-8fcf-3037bfbf2be7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:29.150430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b8521-5ec0-4726-8fcf-3037bfbf2be7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:29.166279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 336/671] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:29.168273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b8521-5ec0-4726-8fcf-3037bfbf2be7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:29.205030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:29.252975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf0ec2-dd16-4d60-98e4-a6d840d50173 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:29.299679 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b8521-5ec0-4726-8fcf-3037bfbf2be7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:29.323324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f4636-118c-41ab-af98-a2b44bd4bcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:29.324933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f4636-118c-41ab-af98-a2b44bd4bcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:29.325719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f4636-118c-41ab-af98-a2b44bd4bcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:29.326509 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f4636-118c-41ab-af98-a2b44bd4bcb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:29.335822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:29.341307 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06ad4a-159e-4271-8d5f-5ffaf549b0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:29.341950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06ad4a-159e-4271-8d5f-5ffaf549b0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:29.342277 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06ad4a-159e-4271-8d5f-5ffaf549b0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:29.348157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06ad4a-159e-4271-8d5f-5ffaf549b0f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:29.356307 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06ad4a-159e-4271-8d5f-5ffaf549b0f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:29.358064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06ad4a-159e-4271-8d5f-5ffaf549b0f4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0e06ad4a-159e-4271-8d5f-5ffaf549b0f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:29.378213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:29.380527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:29.382689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:29.382689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:29.382689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:29.382689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 337/672] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:29.382689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06ad4a-159e-4271-8d5f-5ffaf549b0f4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:29.466091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f4636-118c-41ab-af98-a2b44bd4bcb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:29.467701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f4636-118c-41ab-af98-a2b44bd4bcb7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:29.552751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06ad4a-159e-4271-8d5f-5ffaf549b0f4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:29.578801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:29.598868 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953b59e-b83d-475f-a5cf-4534e33b3b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:29.599506 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953b59e-b83d-475f-a5cf-4534e33b3b21 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:29.599780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953b59e-b83d-475f-a5cf-4534e33b3b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:29.600057 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953b59e-b83d-475f-a5cf-4534e33b3b21 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:29.610651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953b59e-b83d-475f-a5cf-4534e33b3b21 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:29.612135 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953b59e-b83d-475f-a5cf-4534e33b3b21 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:29.625729 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 336/673] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c953b59e-b83d-475f-a5cf-4534e33b3b21 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:29.626856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:29.628617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:29.631530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:29.631530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:29.631530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:29.631530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:29.631530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:29.631530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:29.631530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:29.631530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:29.631530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 08:26:29.631530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:29.633458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 338/674] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:26:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:29.633686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953b59e-b83d-475f-a5cf-4534e33b3b21 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:29.635864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f4636-118c-41ab-af98-a2b44bd4bcb7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:29.692966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:29.762955 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953b59e-b83d-475f-a5cf-4534e33b3b21 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:29.805702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff53a0-8608-41ed-b47e-c2424bc2ebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:29.806364 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff53a0-8608-41ed-b47e-c2424bc2ebe1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:29.806643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff53a0-8608-41ed-b47e-c2424bc2ebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:29.806912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff53a0-8608-41ed-b47e-c2424bc2ebe1 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:29.807586 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff53a0-8608-41ed-b47e-c2424bc2ebe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:29.807889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff53a0-8608-41ed-b47e-c2424bc2ebe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:29.810042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff53a0-8608-41ed-b47e-c2424bc2ebe1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:29.827746 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 339/675] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:29.828759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff53a0-8608-41ed-b47e-c2424bc2ebe1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:29.836751 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f4636-118c-41ab-af98-a2b44bd4bcb7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:29.912014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636e225-4c41-41ec-844d-3ab69e816f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:29.913305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636e225-4c41-41ec-844d-3ab69e816f40 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:29.913945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636e225-4c41-41ec-844d-3ab69e816f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:29.914681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636e225-4c41-41ec-844d-3ab69e816f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:29.935808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636e225-4c41-41ec-844d-3ab69e816f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:29.939963 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636e225-4c41-41ec-844d-3ab69e816f40 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:29.950799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff53a0-8608-41ed-b47e-c2424bc2ebe1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:29.953565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1636e225-4c41-41ec-844d-3ab69e816f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:29.975021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:29.976839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:29.978601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:29.978601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:29.978601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:29.982518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 337/676] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:29.984512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636e225-4c41-41ec-844d-3ab69e816f40 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:29.995120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f2cb3-3199-4ff7-8a7c-a7a3ee0182f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:29.995750 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f2cb3-3199-4ff7-8a7c-a7a3ee0182f8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:29.996031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f2cb3-3199-4ff7-8a7c-a7a3ee0182f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:29.996354 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f2cb3-3199-4ff7-8a7c-a7a3ee0182f8 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:29.997031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f2cb3-3199-4ff7-8a7c-a7a3ee0182f8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:29.997347 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f2cb3-3199-4ff7-8a7c-a7a3ee0182f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:29.998837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f2cb3-3199-4ff7-8a7c-a7a3ee0182f8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:30.022366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 340/677] 10.222.0.44 () {56 vars in 1135 bytes} [Sun Sep 11 08:26:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:30.024291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f2cb3-3199-4ff7-8a7c-a7a3ee0182f8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:30.057031 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:30.165631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f2cb3-3199-4ff7-8a7c-a7a3ee0182f8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:30.208119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4087f3-96f5-4013-b7f5-eca685838372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:30.208746 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4087f3-96f5-4013-b7f5-eca685838372 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:30.209015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4087f3-96f5-4013-b7f5-eca685838372 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:30.209290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4087f3-96f5-4013-b7f5-eca685838372 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:30.209946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4087f3-96f5-4013-b7f5-eca685838372 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:30.210280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4087f3-96f5-4013-b7f5-eca685838372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:30.211691 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4087f3-96f5-4013-b7f5-eca685838372 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:30.212515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636e225-4c41-41ec-844d-3ab69e816f40 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:30.226451 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 341/678] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:30.227425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4087f3-96f5-4013-b7f5-eca685838372 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:30.259761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:30.285642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c4cc1-f66b-4343-8b23-a69d7dfc9b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:30.286276 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c4cc1-f66b-4343-8b23-a69d7dfc9b1d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:30.286560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c4cc1-f66b-4343-8b23-a69d7dfc9b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:30.286836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c4cc1-f66b-4343-8b23-a69d7dfc9b1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:30.294515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c4cc1-f66b-4343-8b23-a69d7dfc9b1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:30.295992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c4cc1-f66b-4343-8b23-a69d7dfc9b1d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0a5c4cc1-f66b-4343-8b23-a69d7dfc9b1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:30.310704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:30.313005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:30.315748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:30.315748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:30.315748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:30.315748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:30.315748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:30.315748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:30.315748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:30.315748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:30.315748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 338/679] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:30.315748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c4cc1-f66b-4343-8b23-a69d7dfc9b1d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:30.426715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:30.427878 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4087f3-96f5-4013-b7f5-eca685838372 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:30.435957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c4cc1-f66b-4343-8b23-a69d7dfc9b1d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:30.477178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0bfe7-1484-42f4-8fa1-33185804f778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:30.477850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0bfe7-1484-42f4-8fa1-33185804f778 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:30.478065 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0bfe7-1484-42f4-8fa1-33185804f778 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:30.478368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0bfe7-1484-42f4-8fa1-33185804f778 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:30.480008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae2448-0740-4061-bc72-3595d8969b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:30.480913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae2448-0740-4061-bc72-3595d8969b8c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:30.481397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae2448-0740-4061-bc72-3595d8969b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:30.481873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae2448-0740-4061-bc72-3595d8969b8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:30.486421 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0bfe7-1484-42f4-8fa1-33185804f778 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:30.487878 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0bfe7-1484-42f4-8fa1-33185804f778 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-abd0bfe7-1484-42f4-8fa1-33185804f778 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:30.500399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:30.501499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:30.502453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:30.502453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:30.502453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:30.502453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:30.502453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:30.502453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:30.502453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:30.502453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:30.502453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 339/680] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:30.502453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0bfe7-1484-42f4-8fa1-33185804f778 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:30.589557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae2448-0740-4061-bc72-3595d8969b8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:30.592666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae2448-0740-4061-bc72-3595d8969b8c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:30.627905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0bfe7-1484-42f4-8fa1-33185804f778 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:30.631845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:30.672665 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b8128-2017-459b-bf8d-32247fa33102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:30.674042 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8128-2017-459b-bf8d-32247fa33102 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:30.674346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8128-2017-459b-bf8d-32247fa33102 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:30.674612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8128-2017-459b-bf8d-32247fa33102 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:30.675323 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b8128-2017-459b-bf8d-32247fa33102 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:30.675624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b8128-2017-459b-bf8d-32247fa33102 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:30.677339 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b8128-2017-459b-bf8d-32247fa33102 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:30.696135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 340/681] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:30.698217 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c1a03-425c-470b-b7e6-1fce0d7f70be None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:30.698546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c1a03-425c-470b-b7e6-1fce0d7f70be None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:30.698767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c1a03-425c-470b-b7e6-1fce0d7f70be None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:30.700729 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 341/682] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:30.701655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c1a03-425c-470b-b7e6-1fce0d7f70be None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:30.718107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:30.739004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 342/683] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:26:30.739004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae2448-0740-4061-bc72-3595d8969b8c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:30.833356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c1a03-425c-470b-b7e6-1fce0d7f70be None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:30.868266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae2448-0740-4061-bc72-3595d8969b8c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:30.887004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742a1da-e265-4fc3-a2c5-52e4e2f0dffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:30.888102 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742a1da-e265-4fc3-a2c5-52e4e2f0dffd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:30.888502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742a1da-e265-4fc3-a2c5-52e4e2f0dffd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:30.888915 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742a1da-e265-4fc3-a2c5-52e4e2f0dffd None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:30.890015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742a1da-e265-4fc3-a2c5-52e4e2f0dffd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:30.891048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742a1da-e265-4fc3-a2c5-52e4e2f0dffd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:30.893809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742a1da-e265-4fc3-a2c5-52e4e2f0dffd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:30.901028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:30.913291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 342/684] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:30.914940 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008abe2e-30b6-4425-b2b7-acba7e834963 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:30.915305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008abe2e-30b6-4425-b2b7-acba7e834963 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:30.915548 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008abe2e-30b6-4425-b2b7-acba7e834963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:30.922886 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69465f60-f12b-4d1c-9da2-2b17cd6963a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:30.924224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69465f60-f12b-4d1c-9da2-2b17cd6963a7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:30.924768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69465f60-f12b-4d1c-9da2-2b17cd6963a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:30.925218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69465f60-f12b-4d1c-9da2-2b17cd6963a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:30.926017 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69465f60-f12b-4d1c-9da2-2b17cd6963a7 None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=58c8502b91ea4ba79e85818727422410) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:30.926405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69465f60-f12b-4d1c-9da2-2b17cd6963a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:30.928034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69465f60-f12b-4d1c-9da2-2b17cd6963a7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:30.935466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 343/685] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:30 2022] GET /identity/v3/role_assignments?user.id=58c8502b91ea4ba79e85818727422410&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:30.936973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69465f60-f12b-4d1c-9da2-2b17cd6963a7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:30.948119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-008abe2e-30b6-4425-b2b7-acba7e834963 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:30.950867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-008abe2e-30b6-4425-b2b7-acba7e834963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:31.079020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:31.094661 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 343/686] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:31.096817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008abe2e-30b6-4425-b2b7-acba7e834963 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:31.111751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69465f60-f12b-4d1c-9da2-2b17cd6963a7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:31.156196 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5ab28-22fe-41d6-93c2-4e7a3f169a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:31.156877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5ab28-22fe-41d6-93c2-4e7a3f169a34 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:31.157214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5ab28-22fe-41d6-93c2-4e7a3f169a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:31.157609 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5ab28-22fe-41d6-93c2-4e7a3f169a34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:31.171071 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5ab28-22fe-41d6-93c2-4e7a3f169a34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:31.173257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5ab28-22fe-41d6-93c2-4e7a3f169a34 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b0a5ab28-22fe-41d6-93c2-4e7a3f169a34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:31.192289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:31.194084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:31.195934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:31.195934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:31.195934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:31.195934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 344/687] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:31.202319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5ab28-22fe-41d6-93c2-4e7a3f169a34 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:31.269431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:31.350325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008abe2e-30b6-4425-b2b7-acba7e834963 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:31.385715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5ab28-22fe-41d6-93c2-4e7a3f169a34 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:31.400079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612423a0-04fd-47ca-92ef-c2e088306cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:31.400697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612423a0-04fd-47ca-92ef-c2e088306cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:31.400969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612423a0-04fd-47ca-92ef-c2e088306cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:31.401231 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612423a0-04fd-47ca-92ef-c2e088306cd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:31.401965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612423a0-04fd-47ca-92ef-c2e088306cd3 None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=92867e15dcf649adad78bf7a29a996f9) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:31.402312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612423a0-04fd-47ca-92ef-c2e088306cd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:31.404472 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612423a0-04fd-47ca-92ef-c2e088306cd3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:31.414355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 344/688] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:31 2022] GET /identity/v3/role_assignments?user.id=92867e15dcf649adad78bf7a29a996f9&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:31.470806 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b10b4-7159-4b56-be22-4ea6d5f45ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:31.473199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b10b4-7159-4b56-be22-4ea6d5f45ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:31.473767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b10b4-7159-4b56-be22-4ea6d5f45ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:31.474274 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b10b4-7159-4b56-be22-4ea6d5f45ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:31.475544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b10b4-7159-4b56-be22-4ea6d5f45ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:31.476084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b10b4-7159-4b56-be22-4ea6d5f45ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:31.478833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b10b4-7159-4b56-be22-4ea6d5f45ee2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:31.508251 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 345/689] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:31.516439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612423a0-04fd-47ca-92ef-c2e088306cd3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:31.654240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:31.702681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b10b4-7159-4b56-be22-4ea6d5f45ee2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:31.732869 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612423a0-04fd-47ca-92ef-c2e088306cd3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:31.806600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d77ec-eb49-4a6c-ad63-a968db67dbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:31.807435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d77ec-eb49-4a6c-ad63-a968db67dbc1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:31.807842 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d77ec-eb49-4a6c-ad63-a968db67dbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:31.808271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d77ec-eb49-4a6c-ad63-a968db67dbc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:31.823394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d77ec-eb49-4a6c-ad63-a968db67dbc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:31.826750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d77ec-eb49-4a6c-ad63-a968db67dbc1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b63d77ec-eb49-4a6c-ad63-a968db67dbc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:31.851239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:31.853027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:31.855624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:31.855624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:31.855624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:31.855624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:31.855624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:31.855624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:31.855624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:31.855624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:31.859657 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 345/690] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:31.869266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d77ec-eb49-4a6c-ad63-a968db67dbc1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:31.912250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:31.920657 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b10b4-7159-4b56-be22-4ea6d5f45ee2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:31.969262 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d6052-353c-46eb-a2a7-24fd7b3b6cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:31.970761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d6052-353c-46eb-a2a7-24fd7b3b6cfc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:31.971236 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d6052-353c-46eb-a2a7-24fd7b3b6cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:31.971654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d6052-353c-46eb-a2a7-24fd7b3b6cfc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:31.972433 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d6052-353c-46eb-a2a7-24fd7b3b6cfc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:31.974111 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d6052-353c-46eb-a2a7-24fd7b3b6cfc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8e7d6052-353c-46eb-a2a7-24fd7b3b6cfc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:31.989672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:31.990907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:31.992168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 346/691] 10.222.0.44 () {56 vars in 1129 bytes} [Sun Sep 11 08:26:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:31.992980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279fecf-551a-49e9-8664-89f86992f2c5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:31.993361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279fecf-551a-49e9-8664-89f86992f2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:31.993588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279fecf-551a-49e9-8664-89f86992f2c5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:32.005440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 347/692] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:32.005440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279fecf-551a-49e9-8664-89f86992f2c5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:32.013998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d77ec-eb49-4a6c-ad63-a968db67dbc1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:32.044926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:32.060378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c399689-5aae-43b3-81f6-bb314ab98a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:32.062212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c399689-5aae-43b3-81f6-bb314ab98a9e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:32.062766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c399689-5aae-43b3-81f6-bb314ab98a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:32.063258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c399689-5aae-43b3-81f6-bb314ab98a9e None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:32.064190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c399689-5aae-43b3-81f6-bb314ab98a9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:32.065144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c399689-5aae-43b3-81f6-bb314ab98a9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:32.067463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c399689-5aae-43b3-81f6-bb314ab98a9e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:32.094346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 346/693] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:32.097304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de6f78-a3d9-4531-be0e-113329d3eb23 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:32.097819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de6f78-a3d9-4531-be0e-113329d3eb23 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:32.098340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de6f78-a3d9-4531-be0e-113329d3eb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:32.132116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-50de6f78-a3d9-4531-be0e-113329d3eb23 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:32.134419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-50de6f78-a3d9-4531-be0e-113329d3eb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:32.144340 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279fecf-551a-49e9-8664-89f86992f2c5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:32.200550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b60ec-e1ba-48c1-b50e-600a67aafff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:32.202346 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b60ec-e1ba-48c1-b50e-600a67aafff4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:32.202851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b60ec-e1ba-48c1-b50e-600a67aafff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:32.203376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b60ec-e1ba-48c1-b50e-600a67aafff4 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:32.205364 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b60ec-e1ba-48c1-b50e-600a67aafff4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:32.206889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b60ec-e1ba-48c1-b50e-600a67aafff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:32.210452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b60ec-e1ba-48c1-b50e-600a67aafff4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:32.225641 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 348/694] 10.222.0.44 () {56 vars in 1124 bytes} [Sun Sep 11 08:26:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:32.225641 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b60ec-e1ba-48c1-b50e-600a67aafff4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:32.258765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 347/695] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:32.258765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de6f78-a3d9-4531-be0e-113329d3eb23 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:32.273491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:32.410182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b60ec-e1ba-48c1-b50e-600a67aafff4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:32.468899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea5048-37e7-48cf-a65e-73af54ab30f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:32.469688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea5048-37e7-48cf-a65e-73af54ab30f1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:32.470891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea5048-37e7-48cf-a65e-73af54ab30f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:32.471450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea5048-37e7-48cf-a65e-73af54ab30f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:32.473604 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea5048-37e7-48cf-a65e-73af54ab30f1 None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=e60a2198022c4f979a6e5e9057558c33) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:32.474033 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea5048-37e7-48cf-a65e-73af54ab30f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:32.476469 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea5048-37e7-48cf-a65e-73af54ab30f1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:32.482820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de6f78-a3d9-4531-be0e-113329d3eb23 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:32.493590 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 349/696] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:32 2022] GET /identity/v3/role_assignments?user.id=e60a2198022c4f979a6e5e9057558c33&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:32.505842 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:32.532783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c00e5a-2cc0-4e08-b1d8-4d5208965271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:32.533775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c00e5a-2cc0-4e08-b1d8-4d5208965271 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:32.533775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c00e5a-2cc0-4e08-b1d8-4d5208965271 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:32.534038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c00e5a-2cc0-4e08-b1d8-4d5208965271 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:32.534716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c00e5a-2cc0-4e08-b1d8-4d5208965271 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:32.535027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c00e5a-2cc0-4e08-b1d8-4d5208965271 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:32.536490 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c00e5a-2cc0-4e08-b1d8-4d5208965271 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:32.544054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 348/697] 10.222.0.44 () {56 vars in 1124 bytes} [Sun Sep 11 08:26:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:32.877374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea5048-37e7-48cf-a65e-73af54ab30f1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:33.003785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:33.019517 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea5048-37e7-48cf-a65e-73af54ab30f1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:33.086686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd69ad0-9f80-4898-aec5-a58b96710c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:33.087341 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd69ad0-9f80-4898-aec5-a58b96710c12 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:33.087620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd69ad0-9f80-4898-aec5-a58b96710c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:33.087892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd69ad0-9f80-4898-aec5-a58b96710c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:33.101580 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd69ad0-9f80-4898-aec5-a58b96710c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:33.103143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd69ad0-9f80-4898-aec5-a58b96710c12 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ccd69ad0-9f80-4898-aec5-a58b96710c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:33.121929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:33.123157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:33.124611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:33.124611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:33.124611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:33.124611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 350/698] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:33.130000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c00e5a-2cc0-4e08-b1d8-4d5208965271 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:33.291112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c00e5a-2cc0-4e08-b1d8-4d5208965271 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:33.349095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8dd96-0d4a-4c72-a259-d858cb9a3634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:33.349737 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8dd96-0d4a-4c72-a259-d858cb9a3634 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:33.350019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8dd96-0d4a-4c72-a259-d858cb9a3634 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:33.350333 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8dd96-0d4a-4c72-a259-d858cb9a3634 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:33.350994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8dd96-0d4a-4c72-a259-d858cb9a3634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:33.351296 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8dd96-0d4a-4c72-a259-d858cb9a3634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:33.352753 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8dd96-0d4a-4c72-a259-d858cb9a3634 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:33.372527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 349/699] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:33.378704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd69ad0-9f80-4898-aec5-a58b96710c12 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:33.548851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd69ad0-9f80-4898-aec5-a58b96710c12 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:33.608141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ef2b4-cadc-43ca-945a-5c169a1db6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:33.608793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ef2b4-cadc-43ca-945a-5c169a1db6a9 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:33.609065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ef2b4-cadc-43ca-945a-5c169a1db6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:33.609336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ef2b4-cadc-43ca-945a-5c169a1db6a9 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:33.610007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ef2b4-cadc-43ca-945a-5c169a1db6a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:33.612122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ef2b4-cadc-43ca-945a-5c169a1db6a9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:33.707263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:33.714223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 351/700] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:26:33 2022] POST /identity/v3/users => generated 254 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:26:34.023439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59de4b-c72a-421c-bff0-924481c3fed0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:34.023933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59de4b-c72a-421c-bff0-924481c3fed0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:34.023933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59de4b-c72a-421c-bff0-924481c3fed0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:34.026300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 350/701] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:34.033344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a26a2-2ae2-41b2-9284-1054a93bd76d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:34.033924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a26a2-2ae2-41b2-9284-1054a93bd76d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:34.034369 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a26a2-2ae2-41b2-9284-1054a93bd76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:34.069856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0a8a26a2-2ae2-41b2-9284-1054a93bd76d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:34.072024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0a8a26a2-2ae2-41b2-9284-1054a93bd76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:34.210827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 352/702] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:34.730109 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59de4b-c72a-421c-bff0-924481c3fed0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:34.815399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:34.930018 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59de4b-c72a-421c-bff0-924481c3fed0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:34.978692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bcfd7-2118-4307-a5f8-e06a376f4c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:34.979303 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bcfd7-2118-4307-a5f8-e06a376f4c30 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:34.979578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bcfd7-2118-4307-a5f8-e06a376f4c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:34.979847 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bcfd7-2118-4307-a5f8-e06a376f4c30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:35.004341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bcfd7-2118-4307-a5f8-e06a376f4c30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-589bcfd7-2118-4307-a5f8-e06a376f4c30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:35.009282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:35.011076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:35.012704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:35.012704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:35.012704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:35.012704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:35.012704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:35.012704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:35.012704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:35.012704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:35.012704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:35.012704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:35.012704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:26:35.012704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:35.014938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 351/703] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:26:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:35.023854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0741e-635b-4ebf-9e97-8416c5c5ec99 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:35.024770 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0741e-635b-4ebf-9e97-8416c5c5ec99 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:35.025522 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0741e-635b-4ebf-9e97-8416c5c5ec99 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:35.028727 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 353/704] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:35.034213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bcfd7-2118-4307-a5f8-e06a376f4c30 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:35.050019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e812940-5fc7-48c6-bd3e-9b02fcc38fa1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:35.050838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e812940-5fc7-48c6-bd3e-9b02fcc38fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:35.050972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e812940-5fc7-48c6-bd3e-9b02fcc38fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:35.104516 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4e812940-5fc7-48c6-bd3e-9b02fcc38fa1 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:35.106589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4e812940-5fc7-48c6-bd3e-9b02fcc38fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:35.301372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bcfd7-2118-4307-a5f8-e06a376f4c30 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:35.304178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:35.319854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 354/705] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 08:26:35.389086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f1992-ffd7-4284-9286-546dffba56cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:35.390110 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f1992-ffd7-4284-9286-546dffba56cf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:35.390618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f1992-ffd7-4284-9286-546dffba56cf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:35.391054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f1992-ffd7-4284-9286-546dffba56cf None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:35.392851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f1992-ffd7-4284-9286-546dffba56cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:35.393400 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f1992-ffd7-4284-9286-546dffba56cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:35.396130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f1992-ffd7-4284-9286-546dffba56cf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:35.421537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 352/706] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:26:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:35.431253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e812940-5fc7-48c6-bd3e-9b02fcc38fa1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:35.557205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:35.632253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e812940-5fc7-48c6-bd3e-9b02fcc38fa1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:35.693711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4911ca-c545-4229-a7be-83ce1ba8c86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:35.694740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4911ca-c545-4229-a7be-83ce1ba8c86e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:35.695172 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4911ca-c545-4229-a7be-83ce1ba8c86e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:35.695586 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4911ca-c545-4229-a7be-83ce1ba8c86e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:35.714600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4911ca-c545-4229-a7be-83ce1ba8c86e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:35.717353 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4911ca-c545-4229-a7be-83ce1ba8c86e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:35.738909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f1992-ffd7-4284-9286-546dffba56cf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ad4911ca-c545-4229-a7be-83ce1ba8c86e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:35.743463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:35.745326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:35.747151 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:35.747151 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:35.747151 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:35.747151 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 355/707] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:35.754048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4911ca-c545-4229-a7be-83ce1ba8c86e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:35.935148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f1992-ffd7-4284-9286-546dffba56cf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:36.026541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b9e3d-5f37-4048-9226-1244beae1831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:36.027532 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b9e3d-5f37-4048-9226-1244beae1831 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:36.027936 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b9e3d-5f37-4048-9226-1244beae1831 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:36.028355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b9e3d-5f37-4048-9226-1244beae1831 None admin] PATH_INFO: `/v3/services/network` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:36.029355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b9e3d-5f37-4048-9226-1244beae1831 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:36.031696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b9e3d-5f37-4048-9226-1244beae1831 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:36.042657 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4911ca-c545-4229-a7be-83ce1ba8c86e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a59b9e3d-5f37-4048-9226-1244beae1831 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:36.053078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:36.054716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:36.058907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 353/708] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:35 2022] GET /identity/v3/services/network => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:36.060138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d5b2c-bdac-482b-b527-8fea4ab4d30e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:36.060138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d5b2c-bdac-482b-b527-8fea4ab4d30e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:36.060138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d5b2c-bdac-482b-b527-8fea4ab4d30e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:36.062697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 354/709] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:36.062697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a05eaf-5052-4b6c-ae35-8595a2d7e8c8 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:36.062925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a05eaf-5052-4b6c-ae35-8595a2d7e8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:36.063374 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a05eaf-5052-4b6c-ae35-8595a2d7e8c8 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:36.064215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:36.066680 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 355/710] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:36.068009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a05eaf-5052-4b6c-ae35-8595a2d7e8c8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:36.096714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9666785-a389-4c9f-acc1-5490ccc8f14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:36.097445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9666785-a389-4c9f-acc1-5490ccc8f14f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:36.097723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9666785-a389-4c9f-acc1-5490ccc8f14f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:36.097992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9666785-a389-4c9f-acc1-5490ccc8f14f None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:36.098670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9666785-a389-4c9f-acc1-5490ccc8f14f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:36.098963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9666785-a389-4c9f-acc1-5490ccc8f14f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:36.100336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9666785-a389-4c9f-acc1-5490ccc8f14f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:36.112263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:36.119420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 356/711] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:36.119420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e926e-2c14-4dd8-a348-8cae3fd3724e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:36.119420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e926e-2c14-4dd8-a348-8cae3fd3724e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:36.119420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e926e-2c14-4dd8-a348-8cae3fd3724e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:36.137258 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0e4e926e-2c14-4dd8-a348-8cae3fd3724e None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:36.138435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0e4e926e-2c14-4dd8-a348-8cae3fd3724e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:36.219143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 357/712] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:36.221434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29956618-048e-4d69-bd30-2d188c46fd9f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:36.221910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29956618-048e-4d69-bd30-2d188c46fd9f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:36.222461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29956618-048e-4d69-bd30-2d188c46fd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:36.258539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-29956618-048e-4d69-bd30-2d188c46fd9f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:36.260853 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-29956618-048e-4d69-bd30-2d188c46fd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:36.262275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a05eaf-5052-4b6c-ae35-8595a2d7e8c8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:36.340002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948c0a8-a326-44fd-9d85-f59d34f073f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:36.342904 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948c0a8-a326-44fd-9d85-f59d34f073f4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:36.343699 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948c0a8-a326-44fd-9d85-f59d34f073f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:36.344463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948c0a8-a326-44fd-9d85-f59d34f073f4 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:36.347220 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948c0a8-a326-44fd-9d85-f59d34f073f4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:36.348108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948c0a8-a326-44fd-9d85-f59d34f073f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:36.352064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948c0a8-a326-44fd-9d85-f59d34f073f4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:36.370876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 356/713] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 08:26:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:36.373845 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948c0a8-a326-44fd-9d85-f59d34f073f4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:36.398939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 358/714] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:36.400787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29956618-048e-4d69-bd30-2d188c46fd9f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:36.476316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:36.614621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:36.643291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948c0a8-a326-44fd-9d85-f59d34f073f4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:36.699466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29956618-048e-4d69-bd30-2d188c46fd9f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:36.715662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114f40e-6944-4885-84fb-18d9a20d109e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:36.716363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114f40e-6944-4885-84fb-18d9a20d109e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:36.716652 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114f40e-6944-4885-84fb-18d9a20d109e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:36.716905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114f40e-6944-4885-84fb-18d9a20d109e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:36.726789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114f40e-6944-4885-84fb-18d9a20d109e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:36.728276 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114f40e-6944-4885-84fb-18d9a20d109e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0114f40e-6944-4885-84fb-18d9a20d109e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:36.742228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:36.744117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:36.746026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:36.746026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:36.746026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:36.746026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:36.746026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:36.746026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:36.746026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:36.746026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:36.746026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:26:36.746026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:36.746026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 357/715] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 08:26:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:36.746026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114f40e-6944-4885-84fb-18d9a20d109e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:36.769587 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd5073-96b9-4f5a-a0cb-bdbef2b91e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:36.770627 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd5073-96b9-4f5a-a0cb-bdbef2b91e1b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:36.771114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd5073-96b9-4f5a-a0cb-bdbef2b91e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:36.771588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd5073-96b9-4f5a-a0cb-bdbef2b91e1b None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:36.772720 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd5073-96b9-4f5a-a0cb-bdbef2b91e1b None admin] RBAC: Adding query filter params (type=network) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:36.773254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd5073-96b9-4f5a-a0cb-bdbef2b91e1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:36.775853 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd5073-96b9-4f5a-a0cb-bdbef2b91e1b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:36.798193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 359/716] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 08:26:36 2022] GET /identity/v3/services?type=network => generated 124 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:36.798193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd5073-96b9-4f5a-a0cb-bdbef2b91e1b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:36.930673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114f40e-6944-4885-84fb-18d9a20d109e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:36.971785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d24614-0ad4-478a-bcba-d83b2d894dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:36.972696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d24614-0ad4-478a-bcba-d83b2d894dfd None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:36.973126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d24614-0ad4-478a-bcba-d83b2d894dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:36.973542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d24614-0ad4-478a-bcba-d83b2d894dfd None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:36.974396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d24614-0ad4-478a-bcba-d83b2d894dfd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:36.976114 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d24614-0ad4-478a-bcba-d83b2d894dfd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:36.998105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:37.009569 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd5073-96b9-4f5a-a0cb-bdbef2b91e1b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:37.067270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef7c39-7ab7-4bab-9a6f-27353d47d386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:37.067947 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef7c39-7ab7-4bab-9a6f-27353d47d386 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:37.068190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef7c39-7ab7-4bab-9a6f-27353d47d386 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:37.068772 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef7c39-7ab7-4bab-9a6f-27353d47d386 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:37.083735 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 358/717] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:26:36 2022] POST /identity/v3/services => generated 251 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:26:37.086592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d24614-0ad4-478a-bcba-d83b2d894dfd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:37.089267 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef7c39-7ab7-4bab-9a6f-27353d47d386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0fef7c39-7ab7-4bab-9a6f-27353d47d386 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:37.098699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:37.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:37.100619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:37.100619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:37.100619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:37.100619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:37.100619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:37.100619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:37.100619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:37.100619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:37.100619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:37.100619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:37.100619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:26:37.100619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:37.106462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 360/718] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:26:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:37.107934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:37.111477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef7c39-7ab7-4bab-9a6f-27353d47d386 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:37.244996 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d24614-0ad4-478a-bcba-d83b2d894dfd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:37.288238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e758-2e90-46b3-8375-ee06777fc177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:37.289265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e758-2e90-46b3-8375-ee06777fc177 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:37.289746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e758-2e90-46b3-8375-ee06777fc177 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:37.290241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e758-2e90-46b3-8375-ee06777fc177 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:37.291231 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804e758-2e90-46b3-8375-ee06777fc177 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:37.291743 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804e758-2e90-46b3-8375-ee06777fc177 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:37.293440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804e758-2e90-46b3-8375-ee06777fc177 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:37.316799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 359/719] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 08:26:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:37.324484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e758-2e90-46b3-8375-ee06777fc177 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:37.357280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef7c39-7ab7-4bab-9a6f-27353d47d386 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:37.413956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:37.425011 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4bf06-7ddf-4949-b644-984af1871051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:37.425955 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4bf06-7ddf-4949-b644-984af1871051 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:37.426448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4bf06-7ddf-4949-b644-984af1871051 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:37.426780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4bf06-7ddf-4949-b644-984af1871051 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:37.430133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4bf06-7ddf-4949-b644-984af1871051 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:37.430133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4bf06-7ddf-4949-b644-984af1871051 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:37.431677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4bf06-7ddf-4949-b644-984af1871051 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:37.444366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 361/720] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:26:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:37.446137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649fc10-5d58-48aa-a1ca-19baa1c1cd64 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:37.446721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649fc10-5d58-48aa-a1ca-19baa1c1cd64 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:37.447246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649fc10-5d58-48aa-a1ca-19baa1c1cd64 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:37.448941 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 362/721] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:37.455212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2834020-6cd6-4d0b-bc4a-003ab6d89b53 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:37.455589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2834020-6cd6-4d0b-bc4a-003ab6d89b53 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:37.455770 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2834020-6cd6-4d0b-bc4a-003ab6d89b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:37.489262 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c2834020-6cd6-4d0b-bc4a-003ab6d89b53 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:37.491247 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c2834020-6cd6-4d0b-bc4a-003ab6d89b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:37.496625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e758-2e90-46b3-8375-ee06777fc177 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:37.517712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:37.554678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5bb71-07e7-4a29-9bc6-cc5dd40ebed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:37.556121 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5bb71-07e7-4a29-9bc6-cc5dd40ebed9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:37.556614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5bb71-07e7-4a29-9bc6-cc5dd40ebed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:37.557093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5bb71-07e7-4a29-9bc6-cc5dd40ebed9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:37.638273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 363/722] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:37.639974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2834020-6cd6-4d0b-bc4a-003ab6d89b53 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:37.795262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5bb71-07e7-4a29-9bc6-cc5dd40ebed9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:37.798212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5bb71-07e7-4a29-9bc6-cc5dd40ebed9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:37.952350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2834020-6cd6-4d0b-bc4a-003ab6d89b53 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:37.971317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:38.024388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1960a7f-3ecd-4e4a-9764-7bdbab00adff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:38.025061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1960a7f-3ecd-4e4a-9764-7bdbab00adff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:38.025329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1960a7f-3ecd-4e4a-9764-7bdbab00adff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:38.025592 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1960a7f-3ecd-4e4a-9764-7bdbab00adff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:38.031196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 360/723] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:37 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 08:26:38.033097 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5bb71-07e7-4a29-9bc6-cc5dd40ebed9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:38.043788 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1960a7f-3ecd-4e4a-9764-7bdbab00adff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:38.045250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1960a7f-3ecd-4e4a-9764-7bdbab00adff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:38.056348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f1960a7f-3ecd-4e4a-9764-7bdbab00adff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:38.067291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:38.073945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:38.076212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:38.076212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:38.076212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:38.076212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 364/724] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:38.076212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1960a7f-3ecd-4e4a-9764-7bdbab00adff None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:38.188677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5bb71-07e7-4a29-9bc6-cc5dd40ebed9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:38.239666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1960a7f-3ecd-4e4a-9764-7bdbab00adff None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:38.243273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c1967-8976-432e-8bd5-0dcc33a8547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:38.244357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c1967-8976-432e-8bd5-0dcc33a8547b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:38.244969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c1967-8976-432e-8bd5-0dcc33a8547b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:38.245550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c1967-8976-432e-8bd5-0dcc33a8547b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:38.268826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c1967-8976-432e-8bd5-0dcc33a8547b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:38.281557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e06f2f-d203-41cb-8364-6b7b0e1640e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6a6c1967-8976-432e-8bd5-0dcc33a8547b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:38.282914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:38.283926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:38.284778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:38.284778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:38.284778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:38.284778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:38.284778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:38.284778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:38.284778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:38.284778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:38.284778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:38.284778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:38.284778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:38.284778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:38.285649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 361/725] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:38.286426 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e06f2f-d203-41cb-8364-6b7b0e1640e4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:38.286952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e06f2f-d203-41cb-8364-6b7b0e1640e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:38.287392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e06f2f-d203-41cb-8364-6b7b0e1640e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:38.287834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c1967-8976-432e-8bd5-0dcc33a8547b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:38.302885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e06f2f-d203-41cb-8364-6b7b0e1640e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:38.304806 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e06f2f-d203-41cb-8364-6b7b0e1640e4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-40e06f2f-d203-41cb-8364-6b7b0e1640e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:38.324481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:38.325446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:38.326479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:38.326479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:38.326479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:38.328959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 365/726] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:38.330619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e06f2f-d203-41cb-8364-6b7b0e1640e4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:38.346608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:38.405421 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:38.436894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c1967-8976-432e-8bd5-0dcc33a8547b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:38.473424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e06f2f-d203-41cb-8364-6b7b0e1640e4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:38.478622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdb3c5-a9f5-42b1-996a-f3a9bd9b28d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:38.479529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdb3c5-a9f5-42b1-996a-f3a9bd9b28d8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:38.479957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdb3c5-a9f5-42b1-996a-f3a9bd9b28d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:38.480375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdb3c5-a9f5-42b1-996a-f3a9bd9b28d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:38.481243 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffdb3c5-a9f5-42b1-996a-f3a9bd9b28d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:38.481706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffdb3c5-a9f5-42b1-996a-f3a9bd9b28d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:38.483444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffdb3c5-a9f5-42b1-996a-f3a9bd9b28d8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:38.513107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 362/727] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:38.514447 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdb3c5-a9f5-42b1-996a-f3a9bd9b28d8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:38.517679 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d6c34-8d3b-4b38-8929-5da31b283d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:38.518969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d6c34-8d3b-4b38-8929-5da31b283d85 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:38.519406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d6c34-8d3b-4b38-8929-5da31b283d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:38.519681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d6c34-8d3b-4b38-8929-5da31b283d85 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:38.521825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d6c34-8d3b-4b38-8929-5da31b283d85 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:38.522140 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d6c34-8d3b-4b38-8929-5da31b283d85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:38.523668 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d6c34-8d3b-4b38-8929-5da31b283d85 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:38.541800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 366/728] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:38.549895 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d6c34-8d3b-4b38-8929-5da31b283d85 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:38.638250 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:38.659966 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdb3c5-a9f5-42b1-996a-f3a9bd9b28d8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:38.693149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d6c34-8d3b-4b38-8929-5da31b283d85 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:38.700553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99637356-1476-4139-83ed-eb50e56078d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:38.701576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99637356-1476-4139-83ed-eb50e56078d7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:38.702038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99637356-1476-4139-83ed-eb50e56078d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:38.702519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99637356-1476-4139-83ed-eb50e56078d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:38.704133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99637356-1476-4139-83ed-eb50e56078d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:38.705185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99637356-1476-4139-83ed-eb50e56078d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:38.707423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99637356-1476-4139-83ed-eb50e56078d7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:38.712230 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:38.726402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 363/729] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:38.728211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99637356-1476-4139-83ed-eb50e56078d7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:38.740215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f563d2e-73fb-4d32-91f2-358c3e8cb1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:38.740713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f563d2e-73fb-4d32-91f2-358c3e8cb1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:38.741065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f563d2e-73fb-4d32-91f2-358c3e8cb1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:38.741409 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f563d2e-73fb-4d32-91f2-358c3e8cb1d8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:38.753111 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f563d2e-73fb-4d32-91f2-358c3e8cb1d8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:38.754768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f563d2e-73fb-4d32-91f2-358c3e8cb1d8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8f563d2e-73fb-4d32-91f2-358c3e8cb1d8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:38.769324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:38.770428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:38.771419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:38.771419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:38.771419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:38.771419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:38.771419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:38.771419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:38.771419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:38.771419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:38.771419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 08:26:38.771419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:38.775804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 367/730] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:26:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:38.777332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f563d2e-73fb-4d32-91f2-358c3e8cb1d8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:38.881722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99637356-1476-4139-83ed-eb50e56078d7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:38.924432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d96c67-fed8-4e6f-a297-2705e4ca50bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:38.925104 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d96c67-fed8-4e6f-a297-2705e4ca50bf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:38.925368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d96c67-fed8-4e6f-a297-2705e4ca50bf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:38.925639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d96c67-fed8-4e6f-a297-2705e4ca50bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:38.938067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d96c67-fed8-4e6f-a297-2705e4ca50bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:38.940943 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f563d2e-73fb-4d32-91f2-358c3e8cb1d8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:38.941254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d96c67-fed8-4e6f-a297-2705e4ca50bf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:38.944426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d5d96c67-fed8-4e6f-a297-2705e4ca50bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:38.969888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:38.972063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:38.973907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:38.973907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:38.973907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:38.975623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 364/731] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:38.976669 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d96c67-fed8-4e6f-a297-2705e4ca50bf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:39.021065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0166e0-113e-410e-af33-8d61072e51b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:39.022619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0166e0-113e-410e-af33-8d61072e51b9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:39.023330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0166e0-113e-410e-af33-8d61072e51b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:39.024019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0166e0-113e-410e-af33-8d61072e51b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:39.038800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0166e0-113e-410e-af33-8d61072e51b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:39.042103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0166e0-113e-410e-af33-8d61072e51b9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5f0166e0-113e-410e-af33-8d61072e51b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:39.072371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:39.074933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:39.076686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:39.076686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:39.076686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:39.076686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:39.076686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:39.076686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:39.076686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:39.076686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:39.076686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 368/732] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:39.080382 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0166e0-113e-410e-af33-8d61072e51b9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:39.124006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:39.139742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d96c67-fed8-4e6f-a297-2705e4ca50bf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:39.210856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a57d5f-34ce-4421-bc52-05b71192bc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:39.211844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a57d5f-34ce-4421-bc52-05b71192bc5f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:39.211844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a57d5f-34ce-4421-bc52-05b71192bc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:39.211844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a57d5f-34ce-4421-bc52-05b71192bc5f None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:39.212799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a57d5f-34ce-4421-bc52-05b71192bc5f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:39.213169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a57d5f-34ce-4421-bc52-05b71192bc5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:39.214882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a57d5f-34ce-4421-bc52-05b71192bc5f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:39.225996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0166e0-113e-410e-af33-8d61072e51b9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:39.244667 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 365/733] 10.222.0.44 () {56 vars in 1135 bytes} [Sun Sep 11 08:26:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:39.245787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a57d5f-34ce-4421-bc52-05b71192bc5f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:39.284619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b0a04-abd7-4c61-a241-f974667d43ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:39.285509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b0a04-abd7-4c61-a241-f974667d43ab None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:39.285704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b0a04-abd7-4c61-a241-f974667d43ab None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:39.286076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b0a04-abd7-4c61-a241-f974667d43ab None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:39.286834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b0a04-abd7-4c61-a241-f974667d43ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:39.287200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b0a04-abd7-4c61-a241-f974667d43ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:39.288683 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b0a04-abd7-4c61-a241-f974667d43ab None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:39.308342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 369/734] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:39.309449 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b0a04-abd7-4c61-a241-f974667d43ab None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:39.385862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:39.449781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:39.474591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b0a04-abd7-4c61-a241-f974667d43ab None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:39.478771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a57d5f-34ce-4421-bc52-05b71192bc5f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:39.550472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215bdb8-9c6a-4274-974b-a16261ad5b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:39.551777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215bdb8-9c6a-4274-974b-a16261ad5b36 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:39.552409 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215bdb8-9c6a-4274-974b-a16261ad5b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:39.552950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215bdb8-9c6a-4274-974b-a16261ad5b36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:39.565292 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04118d00-4f62-4dc3-b208-00a49a265386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:39.566777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04118d00-4f62-4dc3-b208-00a49a265386 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:39.567275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04118d00-4f62-4dc3-b208-00a49a265386 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:39.567766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04118d00-4f62-4dc3-b208-00a49a265386 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:39.569081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04118d00-4f62-4dc3-b208-00a49a265386 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:39.569618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04118d00-4f62-4dc3-b208-00a49a265386 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:39.572431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04118d00-4f62-4dc3-b208-00a49a265386 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:39.597440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 366/735] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:39.599496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04118d00-4f62-4dc3-b208-00a49a265386 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:39.698835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215bdb8-9c6a-4274-974b-a16261ad5b36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:39.700882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215bdb8-9c6a-4274-974b-a16261ad5b36 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:39.770076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04118d00-4f62-4dc3-b208-00a49a265386 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:39.786735 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:39.817596 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263def9-314f-4308-9748-30bf104cc045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:39.818678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263def9-314f-4308-9748-30bf104cc045 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:39.819138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263def9-314f-4308-9748-30bf104cc045 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:39.819591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263def9-314f-4308-9748-30bf104cc045 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:39.833647 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263def9-314f-4308-9748-30bf104cc045 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:39.835587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263def9-314f-4308-9748-30bf104cc045 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-8263def9-314f-4308-9748-30bf104cc045 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:39.850659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:39.852502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:39.855406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:39.855406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:39.855406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:39.855406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:39.855406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:39.855406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:39.855406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:39.855406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:39.855406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 08:26:39.855406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:39.855406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 367/736] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 08:26:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:39.855406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263def9-314f-4308-9748-30bf104cc045 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:39.878308 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:39.895796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 370/737] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:39 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:26:39.898114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cdbc7-9f41-4540-a512-87ba9936a3da None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:39.898502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cdbc7-9f41-4540-a512-87ba9936a3da None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:39.898962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cdbc7-9f41-4540-a512-87ba9936a3da None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:39.901673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 371/738] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:39.902867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cdbc7-9f41-4540-a512-87ba9936a3da None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:40.012725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263def9-314f-4308-9748-30bf104cc045 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:40.047449 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cdbc7-9f41-4540-a512-87ba9936a3da None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:40.057741 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841686c5-c502-4a0d-9b80-e0111cb0bb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:40.058720 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841686c5-c502-4a0d-9b80-e0111cb0bb23 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:40.059156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841686c5-c502-4a0d-9b80-e0111cb0bb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:40.059534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841686c5-c502-4a0d-9b80-e0111cb0bb23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:40.060555 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841686c5-c502-4a0d-9b80-e0111cb0bb23 None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=58c8502b91ea4ba79e85818727422410) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:40.061047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841686c5-c502-4a0d-9b80-e0111cb0bb23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:40.064065 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841686c5-c502-4a0d-9b80-e0111cb0bb23 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:40.073566 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 368/739] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:39 2022] GET /identity/v3/role_assignments?user.id=58c8502b91ea4ba79e85818727422410&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=020bc02dca71485bab2312442adaa179 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:40.075563 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7ab76-b782-4877-97df-37e864f7cafc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:40.076006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7ab76-b782-4877-97df-37e864f7cafc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:40.076341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7ab76-b782-4877-97df-37e864f7cafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:40.104718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0fb7ab76-b782-4877-97df-37e864f7cafc None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:40.106376 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0fb7ab76-b782-4877-97df-37e864f7cafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:40.117058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21fc37-47c8-4527-9e8d-93cb313c8a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:40.118207 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21fc37-47c8-4527-9e8d-93cb313c8a6d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:40.118644 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21fc37-47c8-4527-9e8d-93cb313c8a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:40.119088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21fc37-47c8-4527-9e8d-93cb313c8a6d None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:40.120981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21fc37-47c8-4527-9e8d-93cb313c8a6d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:40.121452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21fc37-47c8-4527-9e8d-93cb313c8a6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:40.124613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21fc37-47c8-4527-9e8d-93cb313c8a6d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:40.147932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:40.166315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 372/740] 10.222.0.44 () {56 vars in 1149 bytes} [Sun Sep 11 08:26:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:40.166315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21fc37-47c8-4527-9e8d-93cb313c8a6d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:40.212068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 369/741] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:40 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:40.212068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac10ae-237e-49ff-b1ff-371f3689ccdf None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:40.212068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac10ae-237e-49ff-b1ff-371f3689ccdf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:40.212068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac10ae-237e-49ff-b1ff-371f3689ccdf None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:40.212068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 370/742] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:40.212068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac10ae-237e-49ff-b1ff-371f3689ccdf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:40.277399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:40.382127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21fc37-47c8-4527-9e8d-93cb313c8a6d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:40.392090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac10ae-237e-49ff-b1ff-371f3689ccdf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:40.432805 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091db7b-f2b5-4daf-836a-fec726e0ef46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:40.434010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091db7b-f2b5-4daf-836a-fec726e0ef46 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:40.434493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091db7b-f2b5-4daf-836a-fec726e0ef46 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:40.434834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091db7b-f2b5-4daf-836a-fec726e0ef46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:40.440366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fdd45-0b06-4fe7-976b-246bf1df4898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:40.441215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fdd45-0b06-4fe7-976b-246bf1df4898 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:40.441592 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fdd45-0b06-4fe7-976b-246bf1df4898 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:40.441964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fdd45-0b06-4fe7-976b-246bf1df4898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:40.458555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fdd45-0b06-4fe7-976b-246bf1df4898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:40.460410 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fdd45-0b06-4fe7-976b-246bf1df4898 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-252fdd45-0b06-4fe7-976b-246bf1df4898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:40.479390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:40.481711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:40.484279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:40.484279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:40.484279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:40.484756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:40.486665 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 373/743] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:40.487854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe349d58-21de-4553-b513-91f120eae812 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:40.488037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe349d58-21de-4553-b513-91f120eae812 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:40.488312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe349d58-21de-4553-b513-91f120eae812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:40.519597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fe349d58-21de-4553-b513-91f120eae812 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:40.521455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fe349d58-21de-4553-b513-91f120eae812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:40.539596 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091db7b-f2b5-4daf-836a-fec726e0ef46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:40.541854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091db7b-f2b5-4daf-836a-fec726e0ef46 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:40.608577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 374/744] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:40 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 08:26:40.609918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe349d58-21de-4553-b513-91f120eae812 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:40.654014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:40.683706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 371/745] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:40 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 08:26:40.693151 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091db7b-f2b5-4daf-836a-fec726e0ef46 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:40.794808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe349d58-21de-4553-b513-91f120eae812 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:40.816391 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:40.836876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbbe23-6626-4a24-882d-c39f30e45d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:40.837749 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbbe23-6626-4a24-882d-c39f30e45d3b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:40.838249 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbbe23-6626-4a24-882d-c39f30e45d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:40.838772 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbbe23-6626-4a24-882d-c39f30e45d3b None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:40.839622 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbbe23-6626-4a24-882d-c39f30e45d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:40.840116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbbe23-6626-4a24-882d-c39f30e45d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:40.842081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbbe23-6626-4a24-882d-c39f30e45d3b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:40.857323 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 375/746] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:40.858648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbbe23-6626-4a24-882d-c39f30e45d3b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:40.875632 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091db7b-f2b5-4daf-836a-fec726e0ef46 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:40.913713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a94855-9125-4e57-ac6a-020a0cc6be9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:40.914692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a94855-9125-4e57-ac6a-020a0cc6be9b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:40.915152 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a94855-9125-4e57-ac6a-020a0cc6be9b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:40.915601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a94855-9125-4e57-ac6a-020a0cc6be9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:40.927830 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a94855-9125-4e57-ac6a-020a0cc6be9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:40.929655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a94855-9125-4e57-ac6a-020a0cc6be9b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-91a94855-9125-4e57-ac6a-020a0cc6be9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:40.946279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:40.947378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:40.948280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:40.948280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:40.948280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:40.948883 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 372/747] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:40.949355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a94855-9125-4e57-ac6a-020a0cc6be9b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:40.974891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:40.985216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbbe23-6626-4a24-882d-c39f30e45d3b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:41.026743 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295eb5f-8070-4655-abb1-23de6554320b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:41.027357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295eb5f-8070-4655-abb1-23de6554320b None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:41.027645 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295eb5f-8070-4655-abb1-23de6554320b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:41.027901 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295eb5f-8070-4655-abb1-23de6554320b None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:41.028560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295eb5f-8070-4655-abb1-23de6554320b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:41.029930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295eb5f-8070-4655-abb1-23de6554320b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:41.063211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a94855-9125-4e57-ac6a-020a0cc6be9b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:41.081548 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:41.100932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714580e-3269-4894-80a9-beca87d82f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:41.101565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714580e-3269-4894-80a9-beca87d82f0f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:41.101843 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714580e-3269-4894-80a9-beca87d82f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:41.102115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714580e-3269-4894-80a9-beca87d82f0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:41.109596 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714580e-3269-4894-80a9-beca87d82f0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:41.111811 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714580e-3269-4894-80a9-beca87d82f0f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:41.118428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 376/748] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:26:40 2022] POST /identity/v3/services => generated 240 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:26:41.118428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295eb5f-8070-4655-abb1-23de6554320b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9714580e-3269-4894-80a9-beca87d82f0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:41.129780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:41.137169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:41.138329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:41.138329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:41.138329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:41.138329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:41.138329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:41.138329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:41.138329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:41.138329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:41.139267 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 373/749] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:41.140415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714580e-3269-4894-80a9-beca87d82f0f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:41.268256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714580e-3269-4894-80a9-beca87d82f0f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:41.314559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c714dc-0022-4119-a9fc-e9e8c0776a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:41.315359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c714dc-0022-4119-a9fc-e9e8c0776a2e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:41.315559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c714dc-0022-4119-a9fc-e9e8c0776a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:41.315732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c714dc-0022-4119-a9fc-e9e8c0776a2e None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:41.316409 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c714dc-0022-4119-a9fc-e9e8c0776a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:41.316713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c714dc-0022-4119-a9fc-e9e8c0776a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:41.318180 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c714dc-0022-4119-a9fc-e9e8c0776a2e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:41.332823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 374/750] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:41.333821 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c714dc-0022-4119-a9fc-e9e8c0776a2e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:41.336320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295eb5f-8070-4655-abb1-23de6554320b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:41.385854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:41.396734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0a45f-040a-4efe-95b7-7416a7b6944a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:41.397626 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0a45f-040a-4efe-95b7-7416a7b6944a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:41.397626 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0a45f-040a-4efe-95b7-7416a7b6944a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:41.397946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0a45f-040a-4efe-95b7-7416a7b6944a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:41.398529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0a45f-040a-4efe-95b7-7416a7b6944a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:41.399906 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0a45f-040a-4efe-95b7-7416a7b6944a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5ed0a45f-040a-4efe-95b7-7416a7b6944a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:41.410328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:41.412344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:41.415377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 377/751] 10.222.0.44 () {56 vars in 1129 bytes} [Sun Sep 11 08:26:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:41.417058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0a45f-040a-4efe-95b7-7416a7b6944a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:41.438320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:41.483520 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c714dc-0022-4119-a9fc-e9e8c0776a2e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:41.525486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f9ba8-1807-4723-8852-08cef46c5ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:41.526109 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f9ba8-1807-4723-8852-08cef46c5ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:41.526424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f9ba8-1807-4723-8852-08cef46c5ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:41.526697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f9ba8-1807-4723-8852-08cef46c5ce2 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:41.527795 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f9ba8-1807-4723-8852-08cef46c5ce2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:41.528114 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f9ba8-1807-4723-8852-08cef46c5ce2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:41.529744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f9ba8-1807-4723-8852-08cef46c5ce2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:41.545456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 375/752] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:41.546416 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f9ba8-1807-4723-8852-08cef46c5ce2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:41.552675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0a45f-040a-4efe-95b7-7416a7b6944a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:41.597162 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a0dda-5442-43c9-a7f5-5dff08faabdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:41.597808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a0dda-5442-43c9-a7f5-5dff08faabdc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:41.598066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a0dda-5442-43c9-a7f5-5dff08faabdc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:41.598425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a0dda-5442-43c9-a7f5-5dff08faabdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:41.607290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a0dda-5442-43c9-a7f5-5dff08faabdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:41.608802 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a0dda-5442-43c9-a7f5-5dff08faabdc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-695a0dda-5442-43c9-a7f5-5dff08faabdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:41.623548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:41.626411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:41.628241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:41.628241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:41.628241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:41.628241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:41.628241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:41.628241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:41.628241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:41.628241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:41.628241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 378/753] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:41.628241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a0dda-5442-43c9-a7f5-5dff08faabdc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:41.658827 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:41.666400 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f9ba8-1807-4723-8852-08cef46c5ce2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:41.706672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfe9d1-8eea-43f3-8581-f0429916434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:41.707455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfe9d1-8eea-43f3-8581-f0429916434a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:41.707951 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfe9d1-8eea-43f3-8581-f0429916434a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:41.708289 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfe9d1-8eea-43f3-8581-f0429916434a None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:41.709196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbfe9d1-8eea-43f3-8581-f0429916434a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:41.709535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbfe9d1-8eea-43f3-8581-f0429916434a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:41.711054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbfe9d1-8eea-43f3-8581-f0429916434a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:41.721770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 376/754] 10.222.0.44 () {56 vars in 1124 bytes} [Sun Sep 11 08:26:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:41.722964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfe9d1-8eea-43f3-8581-f0429916434a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:41.729339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:41.757610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a0dda-5442-43c9-a7f5-5dff08faabdc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:41.798688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1367d5-63ac-464a-96b8-6a92894dfea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:41.799305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1367d5-63ac-464a-96b8-6a92894dfea2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:41.799584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1367d5-63ac-464a-96b8-6a92894dfea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:41.799857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1367d5-63ac-464a-96b8-6a92894dfea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:41.800589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1367d5-63ac-464a-96b8-6a92894dfea2 None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=e60a2198022c4f979a6e5e9057558c33) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:41.800896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1367d5-63ac-464a-96b8-6a92894dfea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:41.803273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1367d5-63ac-464a-96b8-6a92894dfea2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:41.811127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 379/755] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:41 2022] GET /identity/v3/role_assignments?user.id=e60a2198022c4f979a6e5e9057558c33&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=020bc02dca71485bab2312442adaa179 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:41.811989 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1367d5-63ac-464a-96b8-6a92894dfea2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:41.852092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfe9d1-8eea-43f3-8581-f0429916434a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:41.924438 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c4ff7-dac0-404c-9b87-692443d58dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:41.925066 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c4ff7-dac0-404c-9b87-692443d58dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:41.925341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c4ff7-dac0-404c-9b87-692443d58dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:41.925611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c4ff7-dac0-404c-9b87-692443d58dd1 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:41.926326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c4ff7-dac0-404c-9b87-692443d58dd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:41.926629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c4ff7-dac0-404c-9b87-692443d58dd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:41.928096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c4ff7-dac0-404c-9b87-692443d58dd1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:41.963661 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 377/756] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:41.964735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1367d5-63ac-464a-96b8-6a92894dfea2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:41.975905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c4ff7-dac0-404c-9b87-692443d58dd1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:42.028118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:42.058708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81026a7-e762-4d9a-86e7-18556381d66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:42.060378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81026a7-e762-4d9a-86e7-18556381d66a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:42.061721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81026a7-e762-4d9a-86e7-18556381d66a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:42.062313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81026a7-e762-4d9a-86e7-18556381d66a None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:42.064071 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81026a7-e762-4d9a-86e7-18556381d66a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:42.065314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81026a7-e762-4d9a-86e7-18556381d66a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:42.067190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81026a7-e762-4d9a-86e7-18556381d66a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:42.076387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 380/757] 10.222.0.44 () {56 vars in 1124 bytes} [Sun Sep 11 08:26:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:42.082745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81026a7-e762-4d9a-86e7-18556381d66a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:42.089328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:42.201631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c4ff7-dac0-404c-9b87-692443d58dd1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:42.275524 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e26fe1-e79f-45f1-91a1-61fc3616a33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:42.276968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e26fe1-e79f-45f1-91a1-61fc3616a33a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:42.277485 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e26fe1-e79f-45f1-91a1-61fc3616a33a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:42.277956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e26fe1-e79f-45f1-91a1-61fc3616a33a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:42.278961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e26fe1-e79f-45f1-91a1-61fc3616a33a None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=dab0506091034403bf2f0fe5f7538cd6) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:42.279490 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e26fe1-e79f-45f1-91a1-61fc3616a33a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:42.283941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e26fe1-e79f-45f1-91a1-61fc3616a33a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:42.297316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 378/758] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:41 2022] GET /identity/v3/role_assignments?user.id=dab0506091034403bf2f0fe5f7538cd6&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:42.320337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81026a7-e762-4d9a-86e7-18556381d66a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:42.364281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37ce4a-364a-4e42-8c33-789a4f3aa9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:42.365275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37ce4a-364a-4e42-8c33-789a4f3aa9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:42.365783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37ce4a-364a-4e42-8c33-789a4f3aa9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:42.366359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37ce4a-364a-4e42-8c33-789a4f3aa9b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:42.367322 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37ce4a-364a-4e42-8c33-789a4f3aa9b0 None admin] RBAC: Adding query filter params (interface=public, service_id=5fcc9566cbff4c6b9cce839482e980af, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:42.367858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37ce4a-364a-4e42-8c33-789a4f3aa9b0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:42.369535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37ce4a-364a-4e42-8c33-789a4f3aa9b0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:42.380132 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 381/759] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 08:26:42 2022] GET /identity/v3/endpoints?service_id=5fcc9566cbff4c6b9cce839482e980af&interface=public®ion_id=RegionOne => generated 194 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:42.386264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e26fe1-e79f-45f1-91a1-61fc3616a33a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:42.523296 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:42.556174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e26fe1-e79f-45f1-91a1-61fc3616a33a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:42.611001 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44becb5-3974-4497-a127-1a2f2c992d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:42.612014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44becb5-3974-4497-a127-1a2f2c992d4c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:42.612535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44becb5-3974-4497-a127-1a2f2c992d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:42.613009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44becb5-3974-4497-a127-1a2f2c992d4c None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:42.613853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44becb5-3974-4497-a127-1a2f2c992d4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:42.615603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44becb5-3974-4497-a127-1a2f2c992d4c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:42.622704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 379/760] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 08:26:42 2022] GET /identity/v3/services => generated 772 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:43.257814 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f9df2-d67a-422b-96d9-3c1dbf281f9f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:43.258464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f9df2-d67a-422b-96d9-3c1dbf281f9f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:43.258960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f9df2-d67a-422b-96d9-3c1dbf281f9f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:43.260796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 382/761] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:43.267121 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca25083-454e-4fb7-8740-320919de0420 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:43.267753 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca25083-454e-4fb7-8740-320919de0420 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:43.268226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca25083-454e-4fb7-8740-320919de0420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:43.288683 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2ca25083-454e-4fb7-8740-320919de0420 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:43.290764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2ca25083-454e-4fb7-8740-320919de0420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:43.390265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 380/762] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:26:43.762375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f9df2-d67a-422b-96d9-3c1dbf281f9f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:43.845972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:43.885537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f9df2-d67a-422b-96d9-3c1dbf281f9f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:43.925601 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd525b7a-b06a-46cd-8a89-e6441ebef77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:43.926391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd525b7a-b06a-46cd-8a89-e6441ebef77e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:43.926735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd525b7a-b06a-46cd-8a89-e6441ebef77e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:43.927075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd525b7a-b06a-46cd-8a89-e6441ebef77e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:43.940795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd525b7a-b06a-46cd-8a89-e6441ebef77e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-dd525b7a-b06a-46cd-8a89-e6441ebef77e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:43.945949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:43.946985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:43.947830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:43.947830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:43.947830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:43.947830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:43.947830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:43.947830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:43.947830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:43.947830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:43.947830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:43.947830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:43.947830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:26:43.947830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:43.947830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 383/763] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:26:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:43.953734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca25083-454e-4fb7-8740-320919de0420 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:44.075888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:44.114237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca25083-454e-4fb7-8740-320919de0420 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:44.177091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11304d13-7a7f-42e6-ba67-0bd196931e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:44.177727 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11304d13-7a7f-42e6-ba67-0bd196931e0e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:44.178002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11304d13-7a7f-42e6-ba67-0bd196931e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:44.178305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11304d13-7a7f-42e6-ba67-0bd196931e0e None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:44.178988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11304d13-7a7f-42e6-ba67-0bd196931e0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:44.179286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11304d13-7a7f-42e6-ba67-0bd196931e0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:44.180686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11304d13-7a7f-42e6-ba67-0bd196931e0e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:44.191524 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 381/764] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:26:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:44.197986 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd525b7a-b06a-46cd-8a89-e6441ebef77e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:44.292241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98503fb-9e27-4de9-b63d-51914d500f8d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:44.293550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98503fb-9e27-4de9-b63d-51914d500f8d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:44.294052 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98503fb-9e27-4de9-b63d-51914d500f8d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:44.296206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 382/765] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:44.303522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a22fd5-775b-4685-a460-8ec1c990542f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:44.304284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a22fd5-775b-4685-a460-8ec1c990542f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:44.304759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a22fd5-775b-4685-a460-8ec1c990542f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:44.340745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-24a22fd5-775b-4685-a460-8ec1c990542f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:44.342377 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-24a22fd5-775b-4685-a460-8ec1c990542f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:44.449275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd525b7a-b06a-46cd-8a89-e6441ebef77e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:44.504981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 383/766] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:26:44.508261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:44.516724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace7cea-bee8-4bf9-93be-7bbd36d06ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:44.517362 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace7cea-bee8-4bf9-93be-7bbd36d06ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:44.517693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace7cea-bee8-4bf9-93be-7bbd36d06ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:44.517964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace7cea-bee8-4bf9-93be-7bbd36d06ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:44.529643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace7cea-bee8-4bf9-93be-7bbd36d06ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:44.531234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace7cea-bee8-4bf9-93be-7bbd36d06ad8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6ace7cea-bee8-4bf9-93be-7bbd36d06ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:44.551892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:44.552896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:44.553838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:44.553838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:44.553838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:44.564639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 384/767] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:44.575320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a22fd5-775b-4685-a460-8ec1c990542f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:44.675096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:44.713454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a22fd5-775b-4685-a460-8ec1c990542f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:44.766925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e5679-2ca5-4d49-8503-e95f4de127dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:44.767795 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e5679-2ca5-4d49-8503-e95f4de127dc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:44.768247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e5679-2ca5-4d49-8503-e95f4de127dc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:44.768673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e5679-2ca5-4d49-8503-e95f4de127dc None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:44.769480 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e5679-2ca5-4d49-8503-e95f4de127dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:44.769929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e5679-2ca5-4d49-8503-e95f4de127dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:44.771491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e5679-2ca5-4d49-8503-e95f4de127dc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:44.789556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 384/768] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:44.795969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace7cea-bee8-4bf9-93be-7bbd36d06ad8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:45.017805 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e5679-2ca5-4d49-8503-e95f4de127dc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:45.072706 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace7cea-bee8-4bf9-93be-7bbd36d06ad8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:45.176856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ede24d-455d-48a3-8c68-80516a3a8291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:45.177976 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ede24d-455d-48a3-8c68-80516a3a8291 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:45.178519 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ede24d-455d-48a3-8c68-80516a3a8291 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:45.179026 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ede24d-455d-48a3-8c68-80516a3a8291 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:45.196068 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ede24d-455d-48a3-8c68-80516a3a8291 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:45.201127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ede24d-455d-48a3-8c68-80516a3a8291 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-99ede24d-455d-48a3-8c68-80516a3a8291 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:45.255908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:45.256991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:45.257956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:45.257956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:45.257956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:45.257956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:45.257956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:45.257956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:45.257956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:45.257956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:45.257956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:26:45.257956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:45.258526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 385/769] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 08:26:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 463 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:45.264880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ede24d-455d-48a3-8c68-80516a3a8291 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:45.278113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e5679-2ca5-4d49-8503-e95f4de127dc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:45.333333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:45.334461 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:45.352295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29835bd3-d40f-4c96-bec9-ca693c8f99c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:45.353797 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29835bd3-d40f-4c96-bec9-ca693c8f99c8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:45.354549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29835bd3-d40f-4c96-bec9-ca693c8f99c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:45.355589 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29835bd3-d40f-4c96-bec9-ca693c8f99c8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:45.357097 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29835bd3-d40f-4c96-bec9-ca693c8f99c8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:45.360902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29835bd3-d40f-4c96-bec9-ca693c8f99c8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-29835bd3-d40f-4c96-bec9-ca693c8f99c8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:45.386857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:45.388713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:45.390815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 385/770] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:45 2022] GET /identity/v3/services/network => generated 88 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:45.402183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae9775-0a2a-4b01-a071-e4529e47a0c0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:45.402183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae9775-0a2a-4b01-a071-e4529e47a0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:45.402183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae9775-0a2a-4b01-a071-e4529e47a0c0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:45.402183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 386/771] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:45.404928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae9775-0a2a-4b01-a071-e4529e47a0c0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:45.413882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ede24d-455d-48a3-8c68-80516a3a8291 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:45.456682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13d5e2-bcf1-4903-8353-7ac5c804a638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:45.457480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13d5e2-bcf1-4903-8353-7ac5c804a638 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:45.457854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13d5e2-bcf1-4903-8353-7ac5c804a638 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:45.458248 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13d5e2-bcf1-4903-8353-7ac5c804a638 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:45.459079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13d5e2-bcf1-4903-8353-7ac5c804a638 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:45.459483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13d5e2-bcf1-4903-8353-7ac5c804a638 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:45.461010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13d5e2-bcf1-4903-8353-7ac5c804a638 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:45.487764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 386/772] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 08:26:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:45.489439 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27452f8-d9dd-41a6-9230-4ed87c7f911f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:45.489836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27452f8-d9dd-41a6-9230-4ed87c7f911f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:45.490239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27452f8-d9dd-41a6-9230-4ed87c7f911f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:45.513549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d27452f8-d9dd-41a6-9230-4ed87c7f911f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:45.515238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d27452f8-d9dd-41a6-9230-4ed87c7f911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:45.602452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 387/773] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:26:45.603816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27452f8-d9dd-41a6-9230-4ed87c7f911f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:45.636029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae9775-0a2a-4b01-a071-e4529e47a0c0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:45.645164 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:45.675013 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3b605-4afa-4143-8aa7-2d7743222ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:45.675695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3b605-4afa-4143-8aa7-2d7743222ffb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:45.676008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3b605-4afa-4143-8aa7-2d7743222ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:45.676277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3b605-4afa-4143-8aa7-2d7743222ffb None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:45.676905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3b605-4afa-4143-8aa7-2d7743222ffb None admin] RBAC: Adding query filter params (name=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:45.677202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3b605-4afa-4143-8aa7-2d7743222ffb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:45.678640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3b605-4afa-4143-8aa7-2d7743222ffb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:45.688968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 387/774] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 08:26:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:45.690398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae943524-f4ca-4ec4-9ee1-0d791e0846a2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:45.690678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae943524-f4ca-4ec4-9ee1-0d791e0846a2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:45.690947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae943524-f4ca-4ec4-9ee1-0d791e0846a2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:45.692523 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 388/775] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:45.693260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae943524-f4ca-4ec4-9ee1-0d791e0846a2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:45.725713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27452f8-d9dd-41a6-9230-4ed87c7f911f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:45.760736 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:45.763931 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b68f0b-9779-4992-be53-cab48a60cfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:45.764580 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b68f0b-9779-4992-be53-cab48a60cfd0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:45.764858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b68f0b-9779-4992-be53-cab48a60cfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:45.765133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b68f0b-9779-4992-be53-cab48a60cfd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:45.776682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b68f0b-9779-4992-be53-cab48a60cfd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:45.778156 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b68f0b-9779-4992-be53-cab48a60cfd0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-67b68f0b-9779-4992-be53-cab48a60cfd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:45.794467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:45.795493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:45.796840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:45.796840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:45.796840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:45.796840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 388/776] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:45.802211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8369ed8-0599-4d53-b28b-4b51317a3b67 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:45.802211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8369ed8-0599-4d53-b28b-4b51317a3b67 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:45.802211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8369ed8-0599-4d53-b28b-4b51317a3b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:45.811129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae943524-f4ca-4ec4-9ee1-0d791e0846a2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:45.820995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c8369ed8-0599-4d53-b28b-4b51317a3b67 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:45.822468 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c8369ed8-0599-4d53-b28b-4b51317a3b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:45.855122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667eced-eaf8-4eec-bb8c-8c4bcb8573bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:45.855635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667eced-eaf8-4eec-bb8c-8c4bcb8573bd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:45.855906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667eced-eaf8-4eec-bb8c-8c4bcb8573bd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:45.856185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667eced-eaf8-4eec-bb8c-8c4bcb8573bd None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:45.856815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667eced-eaf8-4eec-bb8c-8c4bcb8573bd None admin] RBAC: Adding query filter params (type=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:45.862180 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667eced-eaf8-4eec-bb8c-8c4bcb8573bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:45.862180 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667eced-eaf8-4eec-bb8c-8c4bcb8573bd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:45.869844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 389/777] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 08:26:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:45.870865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667eced-eaf8-4eec-bb8c-8c4bcb8573bd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:45.926810 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 389/778] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:26:45.935164 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8369ed8-0599-4d53-b28b-4b51317a3b67 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:45.950003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:46.042460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667eced-eaf8-4eec-bb8c-8c4bcb8573bd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:46.067737 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:46.083742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76621dd-f60b-4a0a-a726-f5ef95403075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:46.084431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76621dd-f60b-4a0a-a726-f5ef95403075 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:46.085148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76621dd-f60b-4a0a-a726-f5ef95403075 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:46.085148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76621dd-f60b-4a0a-a726-f5ef95403075 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:46.085782 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76621dd-f60b-4a0a-a726-f5ef95403075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:46.086171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76621dd-f60b-4a0a-a726-f5ef95403075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:46.087234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8369ed8-0599-4d53-b28b-4b51317a3b67 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:46.087775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76621dd-f60b-4a0a-a726-f5ef95403075 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:46.106950 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 390/779] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:46.107660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76621dd-f60b-4a0a-a726-f5ef95403075 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:46.165469 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2590bb7-f0cb-4498-9d74-e6f672e5f31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:46.166299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2590bb7-f0cb-4498-9d74-e6f672e5f31f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:46.166578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2590bb7-f0cb-4498-9d74-e6f672e5f31f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:46.167028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2590bb7-f0cb-4498-9d74-e6f672e5f31f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:46.167807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2590bb7-f0cb-4498-9d74-e6f672e5f31f None admin] RBAC: Adding query filter params (interface=public, service_id=0f1805f3e6134ddf92833d0fd1c1654c, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:46.168175 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2590bb7-f0cb-4498-9d74-e6f672e5f31f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:46.169997 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2590bb7-f0cb-4498-9d74-e6f672e5f31f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:46.177662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 390/780] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 08:26:45 2022] GET /identity/v3/endpoints?service_id=0f1805f3e6134ddf92833d0fd1c1654c&interface=public®ion_id=RegionOne => generated 194 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:46.178700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2590bb7-f0cb-4498-9d74-e6f672e5f31f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:46.250752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76621dd-f60b-4a0a-a726-f5ef95403075 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:46.283278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:46.307085 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f2e64-86e9-47d9-ad95-4c7085f0cbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:46.308047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f2e64-86e9-47d9-ad95-4c7085f0cbaf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:46.308474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f2e64-86e9-47d9-ad95-4c7085f0cbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:46.308875 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f2e64-86e9-47d9-ad95-4c7085f0cbaf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:46.313425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2590bb7-f0cb-4498-9d74-e6f672e5f31f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:46.324160 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f2e64-86e9-47d9-ad95-4c7085f0cbaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-078f2e64-86e9-47d9-ad95-4c7085f0cbaf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:46.331707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:46.333781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 391/781] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:46.337351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f2e64-86e9-47d9-ad95-4c7085f0cbaf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:46.371695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d1238-41bb-4983-b4ea-9443ba5c931f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:46.372534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d1238-41bb-4983-b4ea-9443ba5c931f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:46.372983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d1238-41bb-4983-b4ea-9443ba5c931f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:46.373417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d1238-41bb-4983-b4ea-9443ba5c931f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:46.384160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d1238-41bb-4983-b4ea-9443ba5c931f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:46.386732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d1238-41bb-4983-b4ea-9443ba5c931f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:46.401691 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b49d1238-41bb-4983-b4ea-9443ba5c931f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:46.405049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:46.405919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:46.407397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:46.407397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:46.407397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:46.407397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:46.407397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:46.407397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:46.407397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:46.407397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:46.407397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 391/782] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:46.408478 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d1238-41bb-4983-b4ea-9443ba5c931f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:46.490897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f2e64-86e9-47d9-ad95-4c7085f0cbaf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:46.545708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94efad-5af0-4061-9355-5d256ceac1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:46.546692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94efad-5af0-4061-9355-5d256ceac1ea None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:46.547136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94efad-5af0-4061-9355-5d256ceac1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:46.547631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94efad-5af0-4061-9355-5d256ceac1ea None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:46.548567 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94efad-5af0-4061-9355-5d256ceac1ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:46.551002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94efad-5af0-4061-9355-5d256ceac1ea None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:46.559793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 392/783] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 08:26:46 2022] GET /identity/v3/services => generated 772 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:46.563284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94efad-5af0-4061-9355-5d256ceac1ea None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:46.582611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d1238-41bb-4983-b4ea-9443ba5c931f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:46.655290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80298aed-fe55-47c8-bcbc-ce8e8220726f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:46.656761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80298aed-fe55-47c8-bcbc-ce8e8220726f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:46.658087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80298aed-fe55-47c8-bcbc-ce8e8220726f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:46.658802 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80298aed-fe55-47c8-bcbc-ce8e8220726f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:46.663443 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:46.684876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80298aed-fe55-47c8-bcbc-ce8e8220726f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-80298aed-fe55-47c8-bcbc-ce8e8220726f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:46.694488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:46.695798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:46.696625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:46.696625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:46.696625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:46.696625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:46.696625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:46.696625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:46.696625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:46.696625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:46.696625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:46.696625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:46.696625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:26:46.696625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:46.702237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 392/784] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:26:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:46.704840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80298aed-fe55-47c8-bcbc-ce8e8220726f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:46.792747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94efad-5af0-4061-9355-5d256ceac1ea None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:46.795484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:46.852452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7797b51-a9e9-4faf-b6ed-f948cadd8cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:46.854100 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7797b51-a9e9-4faf-b6ed-f948cadd8cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:46.854907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7797b51-a9e9-4faf-b6ed-f948cadd8cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:46.855679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7797b51-a9e9-4faf-b6ed-f948cadd8cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:46.856950 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7797b51-a9e9-4faf-b6ed-f948cadd8cd3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:46.857558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7797b51-a9e9-4faf-b6ed-f948cadd8cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:46.860466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7797b51-a9e9-4faf-b6ed-f948cadd8cd3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:46.877240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80298aed-fe55-47c8-bcbc-ce8e8220726f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:46.884390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 393/785] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:46.884390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7797b51-a9e9-4faf-b6ed-f948cadd8cd3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:46.931321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9199bd5-94a7-401c-a24b-9e07adf43617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:46.932234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9199bd5-94a7-401c-a24b-9e07adf43617 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:46.932714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9199bd5-94a7-401c-a24b-9e07adf43617 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:46.933175 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9199bd5-94a7-401c-a24b-9e07adf43617 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:46.934440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9199bd5-94a7-401c-a24b-9e07adf43617 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:46.934930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9199bd5-94a7-401c-a24b-9e07adf43617 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:46.937566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9199bd5-94a7-401c-a24b-9e07adf43617 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:46.961255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 393/786] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:46.963465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c58d8c-63c0-4e7e-b213-be21cc958568 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:46.963946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c58d8c-63c0-4e7e-b213-be21cc958568 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:46.964412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c58d8c-63c0-4e7e-b213-be21cc958568 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:46.973715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 394/787] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:46.973715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c58d8c-63c0-4e7e-b213-be21cc958568 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:47.054318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:47.066770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7797b51-a9e9-4faf-b6ed-f948cadd8cd3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:47.110508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010c8ea-6b62-48f3-9521-9f0d984b7ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:47.111307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010c8ea-6b62-48f3-9521-9f0d984b7ffb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:47.111711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010c8ea-6b62-48f3-9521-9f0d984b7ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:47.112111 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010c8ea-6b62-48f3-9521-9f0d984b7ffb None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:47.112932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010c8ea-6b62-48f3-9521-9f0d984b7ffb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:47.113365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010c8ea-6b62-48f3-9521-9f0d984b7ffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:47.115446 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010c8ea-6b62-48f3-9521-9f0d984b7ffb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:47.133766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 394/788] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:26:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:47.133766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010c8ea-6b62-48f3-9521-9f0d984b7ffb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:47.148952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c58d8c-63c0-4e7e-b213-be21cc958568 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:47.172719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:47.214033 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba866f2-a56c-4bde-bac8-8646cbf78036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:47.214782 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba866f2-a56c-4bde-bac8-8646cbf78036 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:47.215117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba866f2-a56c-4bde-bac8-8646cbf78036 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:47.215829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba866f2-a56c-4bde-bac8-8646cbf78036 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:47.234010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba866f2-a56c-4bde-bac8-8646cbf78036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:47.236735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba866f2-a56c-4bde-bac8-8646cbf78036 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7ba866f2-a56c-4bde-bac8-8646cbf78036 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:47.264128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:47.268447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:47.269611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:47.269611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:47.269611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:47.270324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 395/789] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:47.272064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858f0f9-f92a-43f5-85b3-dc9974467d60 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:47.272387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858f0f9-f92a-43f5-85b3-dc9974467d60 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:47.272630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858f0f9-f92a-43f5-85b3-dc9974467d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:47.294468 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010c8ea-6b62-48f3-9521-9f0d984b7ffb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:47.301463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4858f0f9-f92a-43f5-85b3-dc9974467d60 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:47.302750 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4858f0f9-f92a-43f5-85b3-dc9974467d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:47.340684 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea69834-fb9e-486d-8c81-072360eb4f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:47.341531 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea69834-fb9e-486d-8c81-072360eb4f16 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:47.341696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea69834-fb9e-486d-8c81-072360eb4f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:47.341962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea69834-fb9e-486d-8c81-072360eb4f16 None admin] PATH_INFO: `/v3/projects/811863e9c94d44a39647800f306505f0/users/58c8502b91ea4ba79e85818727422410/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:47.344067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea69834-fb9e-486d-8c81-072360eb4f16 None admin] RBAC: Authorizing `identity:create_grant(project_id=811863e9c94d44a39647800f306505f0, user_id=58c8502b91ea4ba79e85818727422410, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:47.345540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea69834-fb9e-486d-8c81-072360eb4f16 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:47.411066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 396/790] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:26:47.411066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858f0f9-f92a-43f5-85b3-dc9974467d60 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:47.438890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 395/791] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:26:47 2022] PUT /identity/v3/projects/811863e9c94d44a39647800f306505f0/users/58c8502b91ea4ba79e85818727422410/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:26:47.445957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea69834-fb9e-486d-8c81-072360eb4f16 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:47.478878 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:47.644169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858f0f9-f92a-43f5-85b3-dc9974467d60 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:47.724497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f709-e1e2-4885-8d9e-de6005000a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:47.725144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f709-e1e2-4885-8d9e-de6005000a0b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:47.725426 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f709-e1e2-4885-8d9e-de6005000a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:47.725679 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f709-e1e2-4885-8d9e-de6005000a0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:47.730277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:47.745544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266f709-e1e2-4885-8d9e-de6005000a0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:47.747353 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266f709-e1e2-4885-8d9e-de6005000a0b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:47.760924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea69834-fb9e-486d-8c81-072360eb4f16 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0266f709-e1e2-4885-8d9e-de6005000a0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:47.787885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:47.789974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:47.791704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:47.791704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:47.791704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:47.791704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 397/792] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:47.794309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f709-e1e2-4885-8d9e-de6005000a0b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:47.863411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdf849-0d0c-4f77-92cf-984136e0cd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:47.865433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdf849-0d0c-4f77-92cf-984136e0cd16 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:47.865898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdf849-0d0c-4f77-92cf-984136e0cd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:47.867040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdf849-0d0c-4f77-92cf-984136e0cd16 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:47.868243 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdf849-0d0c-4f77-92cf-984136e0cd16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:47.868780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdf849-0d0c-4f77-92cf-984136e0cd16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:47.872131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdf849-0d0c-4f77-92cf-984136e0cd16 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:47.904997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 396/793] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:47.919241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdf849-0d0c-4f77-92cf-984136e0cd16 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:47.958035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:47.975333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f709-e1e2-4885-8d9e-de6005000a0b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:48.024881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386476d5-3613-4273-8e2e-e5919121cf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:48.026235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386476d5-3613-4273-8e2e-e5919121cf19 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:48.026723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386476d5-3613-4273-8e2e-e5919121cf19 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:48.027197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386476d5-3613-4273-8e2e-e5919121cf19 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:48.028775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386476d5-3613-4273-8e2e-e5919121cf19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:48.030066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386476d5-3613-4273-8e2e-e5919121cf19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:48.033725 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386476d5-3613-4273-8e2e-e5919121cf19 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:48.057062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 398/794] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:48.058483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386476d5-3613-4273-8e2e-e5919121cf19 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:48.109229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdf849-0d0c-4f77-92cf-984136e0cd16 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:48.161078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc577a-aa76-4d08-a78c-df343206b717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:48.162242 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc577a-aa76-4d08-a78c-df343206b717 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:48.162701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc577a-aa76-4d08-a78c-df343206b717 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:48.163189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc577a-aa76-4d08-a78c-df343206b717 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:48.173071 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc577a-aa76-4d08-a78c-df343206b717 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:48.175024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc577a-aa76-4d08-a78c-df343206b717 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-81bc577a-aa76-4d08-a78c-df343206b717 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:48.189077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:48.190094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:48.191278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:48.191278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:48.191278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:48.191278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:48.191278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:48.191278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:48.191278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:48.191278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:48.191278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 08:26:48.191278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:48.193202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 397/795] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 08:26:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:48.194271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc577a-aa76-4d08-a78c-df343206b717 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:48.214928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:48.309039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386476d5-3613-4273-8e2e-e5919121cf19 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:48.328848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc577a-aa76-4d08-a78c-df343206b717 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:48.369478 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6365713-a87b-4732-9401-7e03796bdfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:48.370453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6365713-a87b-4732-9401-7e03796bdfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:48.370873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6365713-a87b-4732-9401-7e03796bdfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:48.371285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6365713-a87b-4732-9401-7e03796bdfc8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:48.372008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6365713-a87b-4732-9401-7e03796bdfc8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:48.373546 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6365713-a87b-4732-9401-7e03796bdfc8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d6365713-a87b-4732-9401-7e03796bdfc8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:48.390114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:48.391210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 08:26:48.392050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:48.392780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 399/796] 10.222.0.44 () {56 vars in 1129 bytes} [Sun Sep 11 08:26:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:48.394997 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6365713-a87b-4732-9401-7e03796bdfc8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:48.396913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c6115-3f92-44ae-a9b8-332f32b119d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:48.397901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c6115-3f92-44ae-a9b8-332f32b119d1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:48.398407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c6115-3f92-44ae-a9b8-332f32b119d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:48.398928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c6115-3f92-44ae-a9b8-332f32b119d1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:48.413376 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c6115-3f92-44ae-a9b8-332f32b119d1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:48.415985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c6115-3f92-44ae-a9b8-332f32b119d1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:48.430588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c21c6115-3f92-44ae-a9b8-332f32b119d1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:48.438370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:48.448036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:48.448981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:48.448981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:48.448981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:48.448981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:48.448981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:48.448981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:48.448981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:48.448981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:48.448981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 08:26:48.448981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:48.448981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 398/797] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:26:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:48.448981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c6115-3f92-44ae-a9b8-332f32b119d1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:48.563569 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:48.579783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6365713-a87b-4732-9401-7e03796bdfc8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:48.624779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c6115-3f92-44ae-a9b8-332f32b119d1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:48.632424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cb0fd-5818-4761-8798-80903afa918f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:48.633046 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cb0fd-5818-4761-8798-80903afa918f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:48.633322 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cb0fd-5818-4761-8798-80903afa918f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:48.633631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cb0fd-5818-4761-8798-80903afa918f None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:48.634585 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cb0fd-5818-4761-8798-80903afa918f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:48.634888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cb0fd-5818-4761-8798-80903afa918f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:48.636386 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cb0fd-5818-4761-8798-80903afa918f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:48.660718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 400/798] 10.222.0.44 () {56 vars in 1149 bytes} [Sun Sep 11 08:26:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:48.662223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cb0fd-5818-4761-8798-80903afa918f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:48.666758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacde41-38cf-4937-bbb1-37d8c541a45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:48.667721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacde41-38cf-4937-bbb1-37d8c541a45d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:48.668198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacde41-38cf-4937-bbb1-37d8c541a45d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:48.668678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacde41-38cf-4937-bbb1-37d8c541a45d None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:48.669523 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacde41-38cf-4937-bbb1-37d8c541a45d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:48.670021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacde41-38cf-4937-bbb1-37d8c541a45d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:48.671634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacde41-38cf-4937-bbb1-37d8c541a45d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:48.678084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 399/799] 10.222.0.44 () {56 vars in 1124 bytes} [Sun Sep 11 08:26:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:48.679354 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacde41-38cf-4937-bbb1-37d8c541a45d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:48.750897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:48.788543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cb0fd-5818-4761-8798-80903afa918f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:48.792413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacde41-38cf-4937-bbb1-37d8c541a45d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:48.824898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:48.831163 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff275696-da7e-43f0-b6e2-9f4987146c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:48.831702 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fd8a5-1f73-46f5-8fdb-8de39b1b2464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:48.831824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff275696-da7e-43f0-b6e2-9f4987146c04 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:48.832049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff275696-da7e-43f0-b6e2-9f4987146c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:48.832320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff275696-da7e-43f0-b6e2-9f4987146c04 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:48.833057 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff275696-da7e-43f0-b6e2-9f4987146c04 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:48.833163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fd8a5-1f73-46f5-8fdb-8de39b1b2464 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:48.833359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff275696-da7e-43f0-b6e2-9f4987146c04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:48.833805 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fd8a5-1f73-46f5-8fdb-8de39b1b2464 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:48.834342 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fd8a5-1f73-46f5-8fdb-8de39b1b2464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:48.834812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff275696-da7e-43f0-b6e2-9f4987146c04 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:48.848058 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54fd8a5-1f73-46f5-8fdb-8de39b1b2464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:48.850000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54fd8a5-1f73-46f5-8fdb-8de39b1b2464 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:48.856772 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 401/800] 10.222.0.44 () {56 vars in 1135 bytes} [Sun Sep 11 08:26:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:48.858614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff275696-da7e-43f0-b6e2-9f4987146c04 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d54fd8a5-1f73-46f5-8fdb-8de39b1b2464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:48.870713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:48.871763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:48.872672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:48.872672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:48.872672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:48.875110 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 400/801] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:48.876552 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fd8a5-1f73-46f5-8fdb-8de39b1b2464 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:48.980915 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff275696-da7e-43f0-b6e2-9f4987146c04 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:49.005219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fd8a5-1f73-46f5-8fdb-8de39b1b2464 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:49.013980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:49.024546 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f6e92-348d-4ba5-b493-cb01b1faa9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:49.028715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f6e92-348d-4ba5-b493-cb01b1faa9bf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:49.028715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f6e92-348d-4ba5-b493-cb01b1faa9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:49.028715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f6e92-348d-4ba5-b493-cb01b1faa9bf None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:49.028715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f6e92-348d-4ba5-b493-cb01b1faa9bf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:49.028715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f6e92-348d-4ba5-b493-cb01b1faa9bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:49.028715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f6e92-348d-4ba5-b493-cb01b1faa9bf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:49.034996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 402/802] 10.222.0.44 () {56 vars in 1124 bytes} [Sun Sep 11 08:26:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:49.035855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f6e92-348d-4ba5-b493-cb01b1faa9bf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:49.068271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5f609-a8bb-4d5e-9a1e-7fec38b6e123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:49.069464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5f609-a8bb-4d5e-9a1e-7fec38b6e123 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:49.069969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5f609-a8bb-4d5e-9a1e-7fec38b6e123 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:49.070465 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5f609-a8bb-4d5e-9a1e-7fec38b6e123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:49.084679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f5f609-a8bb-4d5e-9a1e-7fec38b6e123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:49.086618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f5f609-a8bb-4d5e-9a1e-7fec38b6e123 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f8f5f609-a8bb-4d5e-9a1e-7fec38b6e123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:49.104432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:49.105467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:49.106439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:49.106439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:49.106439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:49.108855 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 401/803] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:49.110254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5f609-a8bb-4d5e-9a1e-7fec38b6e123 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:49.156206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:49.275012 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5f609-a8bb-4d5e-9a1e-7fec38b6e123 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:49.308358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f6e92-348d-4ba5-b493-cb01b1faa9bf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:49.320533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a953c-7320-4c9a-b233-38a3ef74f942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:49.321165 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a953c-7320-4c9a-b233-38a3ef74f942 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:49.321442 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a953c-7320-4c9a-b233-38a3ef74f942 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:49.321713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a953c-7320-4c9a-b233-38a3ef74f942 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:49.322439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a953c-7320-4c9a-b233-38a3ef74f942 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:49.323883 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a953c-7320-4c9a-b233-38a3ef74f942 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:49.378617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205b8cf-d2ca-47dc-8311-dc1106ea9905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:49.380099 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205b8cf-d2ca-47dc-8311-dc1106ea9905 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:49.380832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205b8cf-d2ca-47dc-8311-dc1106ea9905 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:49.381520 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205b8cf-d2ca-47dc-8311-dc1106ea9905 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:49.383009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205b8cf-d2ca-47dc-8311-dc1106ea9905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:49.383772 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205b8cf-d2ca-47dc-8311-dc1106ea9905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:49.386413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205b8cf-d2ca-47dc-8311-dc1106ea9905 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:49.407899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 403/804] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:49.409877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205b8cf-d2ca-47dc-8311-dc1106ea9905 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:49.431012 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 402/805] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:26:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:26:49.432059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a953c-7320-4c9a-b233-38a3ef74f942 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:49.545400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:49.609704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205b8cf-d2ca-47dc-8311-dc1106ea9905 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:49.633692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:49.666413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a953c-7320-4c9a-b233-38a3ef74f942 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:49.667235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95de41e-882b-40a1-a488-7e681f4e7e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:49.667864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95de41e-882b-40a1-a488-7e681f4e7e12 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:49.668145 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95de41e-882b-40a1-a488-7e681f4e7e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:49.668493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95de41e-882b-40a1-a488-7e681f4e7e12 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:49.669031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95de41e-882b-40a1-a488-7e681f4e7e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:49.669329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95de41e-882b-40a1-a488-7e681f4e7e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:49.670781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95de41e-882b-40a1-a488-7e681f4e7e12 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:49.693635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 404/806] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:49.695856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287c93f-289d-424f-8ac7-eb92f97d4602 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:49.696333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287c93f-289d-424f-8ac7-eb92f97d4602 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:49.696791 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287c93f-289d-424f-8ac7-eb92f97d4602 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:49.699339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 405/807] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:49.700730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287c93f-289d-424f-8ac7-eb92f97d4602 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:49.737625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd7498-d202-4de2-a1aa-873f610c8b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:49.738324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd7498-d202-4de2-a1aa-873f610c8b6b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:49.738601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd7498-d202-4de2-a1aa-873f610c8b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:49.738870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd7498-d202-4de2-a1aa-873f610c8b6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:49.747500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd7498-d202-4de2-a1aa-873f610c8b6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:49.749077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd7498-d202-4de2-a1aa-873f610c8b6b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0bdd7498-d202-4de2-a1aa-873f610c8b6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:49.767769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:49.768722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:49.769615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:49.769615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:49.769615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:49.769615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:49.769615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:49.769615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:49.769615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:49.769615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:49.769615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 403/808] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:49.771014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03956ce6-1add-449f-be79-41ac208ae045 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:49.771249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03956ce6-1add-449f-be79-41ac208ae045 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:49.771526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03956ce6-1add-449f-be79-41ac208ae045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:49.796565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-03956ce6-1add-449f-be79-41ac208ae045 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:49.798511 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-03956ce6-1add-449f-be79-41ac208ae045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:49.911278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287c93f-289d-424f-8ac7-eb92f97d4602 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:49.922573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:49.930179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 404/809] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:26:49.930179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03956ce6-1add-449f-be79-41ac208ae045 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:49.973205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb88856-4b3a-48d3-aa24-322e3b217536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:49.974481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb88856-4b3a-48d3-aa24-322e3b217536 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:49.974937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb88856-4b3a-48d3-aa24-322e3b217536 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:49.975425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb88856-4b3a-48d3-aa24-322e3b217536 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:49.983385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb88856-4b3a-48d3-aa24-322e3b217536 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:49.987280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb88856-4b3a-48d3-aa24-322e3b217536 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-afb88856-4b3a-48d3-aa24-322e3b217536 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:50.004154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:50.005264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:50.006161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:50.006161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:50.006161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:50.006161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:50.006161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:50.006161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:50.006161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:50.006161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:50.009376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 406/810] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:50.020996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb88856-4b3a-48d3-aa24-322e3b217536 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:50.116619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:50.140816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03956ce6-1add-449f-be79-41ac208ae045 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:50.174244 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb88856-4b3a-48d3-aa24-322e3b217536 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:50.181021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a024db3-1879-4a62-b8f4-e0bfdc9892b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:50.181635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a024db3-1879-4a62-b8f4-e0bfdc9892b6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:50.181909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a024db3-1879-4a62-b8f4-e0bfdc9892b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:50.182212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a024db3-1879-4a62-b8f4-e0bfdc9892b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:50.182970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a024db3-1879-4a62-b8f4-e0bfdc9892b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:50.183277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a024db3-1879-4a62-b8f4-e0bfdc9892b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:50.185183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a024db3-1879-4a62-b8f4-e0bfdc9892b6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:50.202289 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 405/811] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:50.204788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a024db3-1879-4a62-b8f4-e0bfdc9892b6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:50.234923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08fee2-8ee9-4676-a94c-89ba5f7ad2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:50.235908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08fee2-8ee9-4676-a94c-89ba5f7ad2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:50.236387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08fee2-8ee9-4676-a94c-89ba5f7ad2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:50.236856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08fee2-8ee9-4676-a94c-89ba5f7ad2c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:50.237783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08fee2-8ee9-4676-a94c-89ba5f7ad2c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:50.238317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08fee2-8ee9-4676-a94c-89ba5f7ad2c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:50.239974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08fee2-8ee9-4676-a94c-89ba5f7ad2c7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:50.267133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 407/812] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:50.271270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08fee2-8ee9-4676-a94c-89ba5f7ad2c7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:50.321971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:50.340205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a024db3-1879-4a62-b8f4-e0bfdc9892b6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:50.384784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08424327-131f-448a-bde3-d2ee7ea302dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:50.385466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08424327-131f-448a-bde3-d2ee7ea302dc None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:50.385740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08424327-131f-448a-bde3-d2ee7ea302dc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:50.386006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08424327-131f-448a-bde3-d2ee7ea302dc None admin] PATH_INFO: `/v3/projects/811863e9c94d44a39647800f306505f0/users/dab0506091034403bf2f0fe5f7538cd6/roles/a76c253615fd43a09d6dc924dbe16553` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:50.461023 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08fee2-8ee9-4676-a94c-89ba5f7ad2c7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:50.507210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f44810-8dea-47d5-bbed-a453166d17f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:50.508865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f44810-8dea-47d5-bbed-a453166d17f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:50.509630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f44810-8dea-47d5-bbed-a453166d17f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:50.510446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f44810-8dea-47d5-bbed-a453166d17f8 None admin] PATH_INFO: `/v3/projects/811863e9c94d44a39647800f306505f0/users/e60a2198022c4f979a6e5e9057558c33/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:50.514650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:50.515577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f44810-8dea-47d5-bbed-a453166d17f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=811863e9c94d44a39647800f306505f0, user_id=e60a2198022c4f979a6e5e9057558c33, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:50.518799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f44810-8dea-47d5-bbed-a453166d17f8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:50.523329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08424327-131f-448a-bde3-d2ee7ea302dc None admin] RBAC: Authorizing `identity:create_grant(project_id=811863e9c94d44a39647800f306505f0, user_id=dab0506091034403bf2f0fe5f7538cd6, role_id=a76c253615fd43a09d6dc924dbe16553)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:50.524908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08424327-131f-448a-bde3-d2ee7ea302dc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:50.603265 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 408/813] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:26:50 2022] PUT /identity/v3/projects/811863e9c94d44a39647800f306505f0/users/e60a2198022c4f979a6e5e9057558c33/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:26:50.604651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed4374-262c-4bd7-947b-ff865507ff15 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:50.604937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed4374-262c-4bd7-947b-ff865507ff15 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:50.605225 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed4374-262c-4bd7-947b-ff865507ff15 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:50.606721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 409/814] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:50.607250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed4374-262c-4bd7-947b-ff865507ff15 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:50.654973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 406/815] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:26:50 2022] PUT /identity/v3/projects/811863e9c94d44a39647800f306505f0/users/dab0506091034403bf2f0fe5f7538cd6/roles/a76c253615fd43a09d6dc924dbe16553 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:26:50.657607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f2512-78bd-4b7c-9535-4421da9b61ff None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:50.657992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f2512-78bd-4b7c-9535-4421da9b61ff None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:50.658190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f2512-78bd-4b7c-9535-4421da9b61ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:50.720211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a22f2512-78bd-4b7c-9535-4421da9b61ff None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:50.725016 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a22f2512-78bd-4b7c-9535-4421da9b61ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:50.814230 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:50.839786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed4374-262c-4bd7-947b-ff865507ff15 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:50.889495 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 407/816] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:26:50.899051 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcdc84-210f-4018-923e-044eba156082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:50.899682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcdc84-210f-4018-923e-044eba156082 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:50.899955 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcdc84-210f-4018-923e-044eba156082 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:50.900214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcdc84-210f-4018-923e-044eba156082 None admin] PATH_INFO: `/v3/services/network` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:50.900796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddcdc84-210f-4018-923e-044eba156082 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:50.902163 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddcdc84-210f-4018-923e-044eba156082 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7ddcdc84-210f-4018-923e-044eba156082 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:50.911824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:50.912891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:50.914412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 410/817] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:50 2022] GET /identity/v3/services/network => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:50.919048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f2512-78bd-4b7c-9535-4421da9b61ff None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:51.038034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f2512-78bd-4b7c-9535-4421da9b61ff None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:51.061168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:51.082784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf3e15-651a-4558-b2d8-d3c1db6c2872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:51.083819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf3e15-651a-4558-b2d8-d3c1db6c2872 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:51.084336 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf3e15-651a-4558-b2d8-d3c1db6c2872 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:51.084826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf3e15-651a-4558-b2d8-d3c1db6c2872 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:51.085699 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf3e15-651a-4558-b2d8-d3c1db6c2872 None admin] RBAC: Adding query filter params (name=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:51.086450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf3e15-651a-4558-b2d8-d3c1db6c2872 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:51.088407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf3e15-651a-4558-b2d8-d3c1db6c2872 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:51.094776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 408/818] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 08:26:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:51.101243 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcdc84-210f-4018-923e-044eba156082 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:51.327914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcdc84-210f-4018-923e-044eba156082 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:51.347814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf3e15-651a-4558-b2d8-d3c1db6c2872 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:51.400539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68197bbb-7dc9-472e-92ce-7de901e203bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:51.401161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68197bbb-7dc9-472e-92ce-7de901e203bd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:51.401431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68197bbb-7dc9-472e-92ce-7de901e203bd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:51.401859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68197bbb-7dc9-472e-92ce-7de901e203bd None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:51.402539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68197bbb-7dc9-472e-92ce-7de901e203bd None admin] RBAC: Adding query filter params (type=network) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:51.402838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68197bbb-7dc9-472e-92ce-7de901e203bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:51.404257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68197bbb-7dc9-472e-92ce-7de901e203bd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:51.418816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 411/819] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 08:26:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:51.427399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68197bbb-7dc9-472e-92ce-7de901e203bd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:51.482556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:51.591282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf3e15-651a-4558-b2d8-d3c1db6c2872 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:51.634887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb237c6-4d4e-45c9-8df0-27bdd800e75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:51.635972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb237c6-4d4e-45c9-8df0-27bdd800e75d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:51.636464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb237c6-4d4e-45c9-8df0-27bdd800e75d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:51.636970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb237c6-4d4e-45c9-8df0-27bdd800e75d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:51.649893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68197bbb-7dc9-472e-92ce-7de901e203bd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:51.650449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb237c6-4d4e-45c9-8df0-27bdd800e75d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-dcb237c6-4d4e-45c9-8df0-27bdd800e75d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:51.656207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:51.657799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:51.658776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:51.658776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:51.658776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:51.658776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:51.658776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:51.658776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:51.658776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:51.658776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:51.658776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:51.658776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:51.658776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:26:51.658776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:51.665783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 409/820] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:26:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:51.668634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb237c6-4d4e-45c9-8df0-27bdd800e75d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:51.726018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3b32d-ac4c-414f-a13d-8ef14fb70a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:51.727281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3b32d-ac4c-414f-a13d-8ef14fb70a6d None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:51.728925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3b32d-ac4c-414f-a13d-8ef14fb70a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:51.730636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3b32d-ac4c-414f-a13d-8ef14fb70a6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:51.733833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3b32d-ac4c-414f-a13d-8ef14fb70a6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:51.735685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3b32d-ac4c-414f-a13d-8ef14fb70a6d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:51.751536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:51.862358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb237c6-4d4e-45c9-8df0-27bdd800e75d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:51.863637 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 412/821] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:26:51 2022] POST /identity/v3/endpoints => generated 337 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:26:51.933219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e95ca4-ffbc-4fed-a66f-9472741be977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:51.934244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e95ca4-ffbc-4fed-a66f-9472741be977 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:51.934708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e95ca4-ffbc-4fed-a66f-9472741be977 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:51.935152 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e95ca4-ffbc-4fed-a66f-9472741be977 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:51.936344 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e95ca4-ffbc-4fed-a66f-9472741be977 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:51.936850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e95ca4-ffbc-4fed-a66f-9472741be977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:51.939519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e95ca4-ffbc-4fed-a66f-9472741be977 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:51.955887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 410/822] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:26:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:51.961923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3b32d-ac4c-414f-a13d-8ef14fb70a6d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:52.097664 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:52.148703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3b32d-ac4c-414f-a13d-8ef14fb70a6d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:52.168327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e64710-c031-45e5-9b16-ec2de17d1bba None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:52.168812 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e64710-c031-45e5-9b16-ec2de17d1bba None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:52.169293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e64710-c031-45e5-9b16-ec2de17d1bba None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:52.173761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 411/823] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:52.180808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1c03e-d388-4e9f-9d57-abadf37a01c6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:52.181317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1c03e-d388-4e9f-9d57-abadf37a01c6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:52.181734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1c03e-d388-4e9f-9d57-abadf37a01c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:52.212174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-10a1c03e-d388-4e9f-9d57-abadf37a01c6 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:52.216071 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-10a1c03e-d388-4e9f-9d57-abadf37a01c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:52.225592 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0008a-6fbd-4715-93f1-caed25bf00e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:52.226710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0008a-6fbd-4715-93f1-caed25bf00e3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:52.227227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0008a-6fbd-4715-93f1-caed25bf00e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:52.227690 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0008a-6fbd-4715-93f1-caed25bf00e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:52.240021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0008a-6fbd-4715-93f1-caed25bf00e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:52.241972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0008a-6fbd-4715-93f1-caed25bf00e3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-25f0008a-6fbd-4715-93f1-caed25bf00e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:52.258285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:52.259331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:52.260230 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:52.260230 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:52.260230 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:52.262510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 413/824] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:52.269906 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0008a-6fbd-4715-93f1-caed25bf00e3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:52.356101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 412/825] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:26:52.452419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0008a-6fbd-4715-93f1-caed25bf00e3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:52.498742 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:52.518813 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb077bd-9f98-43f7-b18c-967d37ef6dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:52.520275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb077bd-9f98-43f7-b18c-967d37ef6dfb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:52.521026 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb077bd-9f98-43f7-b18c-967d37ef6dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:52.521741 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb077bd-9f98-43f7-b18c-967d37ef6dfb None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:52.523538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb077bd-9f98-43f7-b18c-967d37ef6dfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:52.524374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb077bd-9f98-43f7-b18c-967d37ef6dfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:52.527359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb077bd-9f98-43f7-b18c-967d37ef6dfb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:52.553059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 414/826] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:52.559930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1c03e-d388-4e9f-9d57-abadf37a01c6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:52.597263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:52.683698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb077bd-9f98-43f7-b18c-967d37ef6dfb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:52.690444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1c03e-d388-4e9f-9d57-abadf37a01c6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:52.731660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c343-2b88-4ca5-8872-809e4fa3ead5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:52.732795 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c343-2b88-4ca5-8872-809e4fa3ead5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:52.733364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c343-2b88-4ca5-8872-809e4fa3ead5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:52.733636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c343-2b88-4ca5-8872-809e4fa3ead5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:52.746788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1c343-2b88-4ca5-8872-809e4fa3ead5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:52.749574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1c343-2b88-4ca5-8872-809e4fa3ead5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a2b1c343-2b88-4ca5-8872-809e4fa3ead5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:52.777890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:52.779621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:52.780794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:52.780794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:52.780794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:52.780794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:52.780794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:52.780794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:52.780794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:52.780794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:52.780794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:26:52.780794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:52.781336 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 413/827] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 08:26:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:52.789107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c343-2b88-4ca5-8872-809e4fa3ead5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:52.832136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb077bd-9f98-43f7-b18c-967d37ef6dfb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:52.886672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071693e-8f83-4902-9a12-c1e4ae3fba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:52.888083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071693e-8f83-4902-9a12-c1e4ae3fba7b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:52.888926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071693e-8f83-4902-9a12-c1e4ae3fba7b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:52.889425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071693e-8f83-4902-9a12-c1e4ae3fba7b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:52.890250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071693e-8f83-4902-9a12-c1e4ae3fba7b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:52.892401 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071693e-8f83-4902-9a12-c1e4ae3fba7b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6071693e-8f83-4902-9a12-c1e4ae3fba7b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:52.903861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:52.904928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 08:26:52.905787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:52.908838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 415/828] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:26:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:52.920709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071693e-8f83-4902-9a12-c1e4ae3fba7b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:52.945163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:52.965114 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c343-2b88-4ca5-8872-809e4fa3ead5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:52.977955 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:53.037852 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071693e-8f83-4902-9a12-c1e4ae3fba7b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:53.038337 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf688c-be04-4f3d-8417-c2e12cf0b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:53.039357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf688c-be04-4f3d-8417-c2e12cf0b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:53.039843 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf688c-be04-4f3d-8417-c2e12cf0b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:53.040609 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf688c-be04-4f3d-8417-c2e12cf0b7c3 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:53.042447 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf688c-be04-4f3d-8417-c2e12cf0b7c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:53.042677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf688c-be04-4f3d-8417-c2e12cf0b7c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:53.045331 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf688c-be04-4f3d-8417-c2e12cf0b7c3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:53.082770 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84836cc5-bd5e-4312-8feb-6b71e2e8e1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:53.083843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84836cc5-bd5e-4312-8feb-6b71e2e8e1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:53.084356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84836cc5-bd5e-4312-8feb-6b71e2e8e1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:53.084845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84836cc5-bd5e-4312-8feb-6b71e2e8e1a0 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:53.085729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84836cc5-bd5e-4312-8feb-6b71e2e8e1a0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:53.090209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 414/829] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 08:26:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:53.094199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84836cc5-bd5e-4312-8feb-6b71e2e8e1a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:53.097034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84836cc5-bd5e-4312-8feb-6b71e2e8e1a0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:53.098411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf688c-be04-4f3d-8417-c2e12cf0b7c3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:53.108196 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 416/830] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:26:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:53.116358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84836cc5-bd5e-4312-8feb-6b71e2e8e1a0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:53.228224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84836cc5-bd5e-4312-8feb-6b71e2e8e1a0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:53.242298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:53.264660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717e0be-bd87-47d2-a849-8122e33bb905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:53.265415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717e0be-bd87-47d2-a849-8122e33bb905 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:53.265766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717e0be-bd87-47d2-a849-8122e33bb905 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:53.266188 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717e0be-bd87-47d2-a849-8122e33bb905 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:53.266876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717e0be-bd87-47d2-a849-8122e33bb905 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:53.267182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717e0be-bd87-47d2-a849-8122e33bb905 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:53.268542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717e0be-bd87-47d2-a849-8122e33bb905 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:53.278200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 417/831] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:26:53 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:53.278200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3a04f-0cfd-45ab-a77d-bb54e7b63287 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:53.278200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3a04f-0cfd-45ab-a77d-bb54e7b63287 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:53.278200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3a04f-0cfd-45ab-a77d-bb54e7b63287 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:53.278200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 418/832] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:53.286197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e231b3d-e6c2-4985-b8cb-de0d20894388 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:53.286449 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e231b3d-e6c2-4985-b8cb-de0d20894388 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:53.286702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e231b3d-e6c2-4985-b8cb-de0d20894388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:53.311030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2e231b3d-e6c2-4985-b8cb-de0d20894388 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:53.311515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf688c-be04-4f3d-8417-c2e12cf0b7c3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:53.312251 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2e231b3d-e6c2-4985-b8cb-de0d20894388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:53.398718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111cdfa-e944-443a-8d7e-3a066d75da02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:53.400765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111cdfa-e944-443a-8d7e-3a066d75da02 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:53.401864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111cdfa-e944-443a-8d7e-3a066d75da02 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:53.402606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111cdfa-e944-443a-8d7e-3a066d75da02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:53.412609 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 419/833] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:26:53.428404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:53.485574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111cdfa-e944-443a-8d7e-3a066d75da02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:53.487227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111cdfa-e944-443a-8d7e-3a066d75da02 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:53.616067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 415/834] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:53 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:26:53.624555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e231b3d-e6c2-4985-b8cb-de0d20894388 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:53.649340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e592e-97a4-435a-b28d-3d3eac890e53 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:53.649703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e592e-97a4-435a-b28d-3d3eac890e53 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:53.649876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e592e-97a4-435a-b28d-3d3eac890e53 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:53.651518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 416/835] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:53.657917 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dd771-9b31-4149-a4a9-5a6190e77c4a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:53.658537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dd771-9b31-4149-a4a9-5a6190e77c4a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:53.659014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dd771-9b31-4149-a4a9-5a6190e77c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:53.681182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5b5dd771-9b31-4149-a4a9-5a6190e77c4a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:53.683568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5b5dd771-9b31-4149-a4a9-5a6190e77c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:53.748992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:53.757224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e231b3d-e6c2-4985-b8cb-de0d20894388 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:53.767373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:53.790383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 417/836] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:26:53.808011 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4710cc3-625e-45f1-814e-70e0416efaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:53.809118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4710cc3-625e-45f1-814e-70e0416efaaa None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:53.809687 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4710cc3-625e-45f1-814e-70e0416efaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:53.810280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4710cc3-625e-45f1-814e-70e0416efaaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:53.823142 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4710cc3-625e-45f1-814e-70e0416efaaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:53.825129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4710cc3-625e-45f1-814e-70e0416efaaa None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f4710cc3-625e-45f1-814e-70e0416efaaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:53.844919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:53.845925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:53.846837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:53.846837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:53.846837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:53.849135 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 420/837] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:53.860976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5dd771-9b31-4149-a4a9-5a6190e77c4a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:53.925191 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4710cc3-625e-45f1-814e-70e0416efaaa None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:54.042848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5dd771-9b31-4149-a4a9-5a6190e77c4a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:54.073878 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4710cc3-625e-45f1-814e-70e0416efaaa None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:54.084378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c201a5-5cab-41cd-b0da-5b23b2dfd21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:54.085404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c201a5-5cab-41cd-b0da-5b23b2dfd21f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:54.085885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c201a5-5cab-41cd-b0da-5b23b2dfd21f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:54.086405 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c201a5-5cab-41cd-b0da-5b23b2dfd21f None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:54.087332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c201a5-5cab-41cd-b0da-5b23b2dfd21f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:54.087845 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c201a5-5cab-41cd-b0da-5b23b2dfd21f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:54.089496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c201a5-5cab-41cd-b0da-5b23b2dfd21f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:54.105395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 418/838] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:54.106742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c201a5-5cab-41cd-b0da-5b23b2dfd21f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:54.135241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c9aaf-d160-48a3-8f33-32bd794834d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:54.136210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c9aaf-d160-48a3-8f33-32bd794834d8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:54.136683 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c9aaf-d160-48a3-8f33-32bd794834d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:54.137179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c9aaf-d160-48a3-8f33-32bd794834d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:54.149378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c9aaf-d160-48a3-8f33-32bd794834d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-696c9aaf-d160-48a3-8f33-32bd794834d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:54.155400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:54.156437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:54.157860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:54.157860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:54.157860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:54.157860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:54.157860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:54.157860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:54.157860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:54.157860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:54.157860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:54.157860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:54.157860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:26:54.157860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:54.158881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 421/839] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:26:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:54.159740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c9aaf-d160-48a3-8f33-32bd794834d8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:54.174869 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:54.186712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:54.268265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c201a5-5cab-41cd-b0da-5b23b2dfd21f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:54.310459 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c9aaf-d160-48a3-8f33-32bd794834d8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:54.322478 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32a85a-b96b-4a02-8d27-6c7cc6c6fa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:54.323261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32a85a-b96b-4a02-8d27-6c7cc6c6fa68 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:54.324322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32a85a-b96b-4a02-8d27-6c7cc6c6fa68 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:54.324700 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32a85a-b96b-4a02-8d27-6c7cc6c6fa68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:54.344026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32a85a-b96b-4a02-8d27-6c7cc6c6fa68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-bf32a85a-b96b-4a02-8d27-6c7cc6c6fa68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:26:54.352670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:54.354787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 419/840] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:26:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:54.356714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32a85a-b96b-4a02-8d27-6c7cc6c6fa68 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:54.377568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9538ca-2b59-4eae-a34a-fb13c31d7d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:54.378774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9538ca-2b59-4eae-a34a-fb13c31d7d34 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:54.379329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9538ca-2b59-4eae-a34a-fb13c31d7d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:54.379950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9538ca-2b59-4eae-a34a-fb13c31d7d34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:54.392435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9538ca-2b59-4eae-a34a-fb13c31d7d34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:54.394874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9538ca-2b59-4eae-a34a-fb13c31d7d34 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ed9538ca-2b59-4eae-a34a-fb13c31d7d34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:54.414349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:54.416167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:54.417936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:54.417936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:54.417936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:54.417936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:54.417936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:54.417936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:54.417936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:54.417936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:54.423043 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 422/841] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:54.424215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9538ca-2b59-4eae-a34a-fb13c31d7d34 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:54.524676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:54.530454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32a85a-b96b-4a02-8d27-6c7cc6c6fa68 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:54.566430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:54.586361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f2420-4583-4b16-ada9-77bdc0eb95fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:54.587340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f2420-4583-4b16-ada9-77bdc0eb95fa None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:54.588392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f2420-4583-4b16-ada9-77bdc0eb95fa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:54.588859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f2420-4583-4b16-ada9-77bdc0eb95fa None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:54.589744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f2420-4583-4b16-ada9-77bdc0eb95fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:54.590926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f2420-4583-4b16-ada9-77bdc0eb95fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:54.592520 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f2420-4583-4b16-ada9-77bdc0eb95fa None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:54.610419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 420/842] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:26:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:54.610419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f2420-4583-4b16-ada9-77bdc0eb95fa None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:54.618887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9538ca-2b59-4eae-a34a-fb13c31d7d34 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:54.686581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d4dd6-5ca8-4e39-8920-94c25fe6446c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:54.687474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d4dd6-5ca8-4e39-8920-94c25fe6446c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:54.688000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d4dd6-5ca8-4e39-8920-94c25fe6446c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:54.688466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d4dd6-5ca8-4e39-8920-94c25fe6446c None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:54.689670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83d4dd6-5ca8-4e39-8920-94c25fe6446c None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:54.690226 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83d4dd6-5ca8-4e39-8920-94c25fe6446c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:54.693197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83d4dd6-5ca8-4e39-8920-94c25fe6446c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:54.714558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 423/843] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:26:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:54.715880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d4dd6-5ca8-4e39-8920-94c25fe6446c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:54.735170 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f2420-4583-4b16-ada9-77bdc0eb95fa None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:54.776514 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b892aa-fc3a-4bf5-83bd-aa2be4ef8bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:54.777129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b892aa-fc3a-4bf5-83bd-aa2be4ef8bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:54.777415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b892aa-fc3a-4bf5-83bd-aa2be4ef8bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:54.777662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b892aa-fc3a-4bf5-83bd-aa2be4ef8bd5 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:54.778469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b892aa-fc3a-4bf5-83bd-aa2be4ef8bd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:54.778772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b892aa-fc3a-4bf5-83bd-aa2be4ef8bd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:54.780215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b892aa-fc3a-4bf5-83bd-aa2be4ef8bd5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:54.799938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 421/844] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:54.799938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b892aa-fc3a-4bf5-83bd-aa2be4ef8bd5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:54.822685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:54.865780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d4dd6-5ca8-4e39-8920-94c25fe6446c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:54.907462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a5e7f-6b26-4e47-81bf-8e27ff33f27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:54.908371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a5e7f-6b26-4e47-81bf-8e27ff33f27b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:54.908651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a5e7f-6b26-4e47-81bf-8e27ff33f27b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:54.908921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a5e7f-6b26-4e47-81bf-8e27ff33f27b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:54.920812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:54.925013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a5e7f-6b26-4e47-81bf-8e27ff33f27b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:54.926634 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a5e7f-6b26-4e47-81bf-8e27ff33f27b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-629a5e7f-6b26-4e47-81bf-8e27ff33f27b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:54.945088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:54.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:54.956290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:54.956290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:54.956290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:54.956290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 424/845] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:54.956712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b892aa-fc3a-4bf5-83bd-aa2be4ef8bd5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:54.956836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a5e7f-6b26-4e47-81bf-8e27ff33f27b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:54.999886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21d55f-ed59-4cc4-b8fc-4f3f9cd7d577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:55.000921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21d55f-ed59-4cc4-b8fc-4f3f9cd7d577 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:55.001332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21d55f-ed59-4cc4-b8fc-4f3f9cd7d577 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:55.001743 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21d55f-ed59-4cc4-b8fc-4f3f9cd7d577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:55.015875 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21d55f-ed59-4cc4-b8fc-4f3f9cd7d577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:55.017991 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21d55f-ed59-4cc4-b8fc-4f3f9cd7d577 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7c21d55f-ed59-4cc4-b8fc-4f3f9cd7d577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:55.037175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:55.039026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:55.041633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:55.041633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:55.041633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:55.041633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 422/846] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:55.041633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21d55f-ed59-4cc4-b8fc-4f3f9cd7d577 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:55.112950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a5e7f-6b26-4e47-81bf-8e27ff33f27b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:55.153739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401536a-5b4a-4682-b2d4-7fd93b1e60a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:55.154406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401536a-5b4a-4682-b2d4-7fd93b1e60a5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:55.154682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401536a-5b4a-4682-b2d4-7fd93b1e60a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:55.154947 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401536a-5b4a-4682-b2d4-7fd93b1e60a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:55.155668 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401536a-5b4a-4682-b2d4-7fd93b1e60a5 None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=58c8502b91ea4ba79e85818727422410) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:55.155967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401536a-5b4a-4682-b2d4-7fd93b1e60a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:55.157452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401536a-5b4a-4682-b2d4-7fd93b1e60a5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:55.159637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:55.164055 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 425/847] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:54 2022] GET /identity/v3/role_assignments?user.id=58c8502b91ea4ba79e85818727422410&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=020bc02dca71485bab2312442adaa179 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:55.164660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401536a-5b4a-4682-b2d4-7fd93b1e60a5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:55.214372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21d55f-ed59-4cc4-b8fc-4f3f9cd7d577 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:55.249502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:55.265683 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454d6f7-6209-4ffb-8060-00c00ce40d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:55.267858 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454d6f7-6209-4ffb-8060-00c00ce40d0e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:55.268339 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454d6f7-6209-4ffb-8060-00c00ce40d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:55.269074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454d6f7-6209-4ffb-8060-00c00ce40d0e None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:55.269942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454d6f7-6209-4ffb-8060-00c00ce40d0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:55.270460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454d6f7-6209-4ffb-8060-00c00ce40d0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:55.273352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454d6f7-6209-4ffb-8060-00c00ce40d0e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:55.292639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 423/848] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:55.300230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454d6f7-6209-4ffb-8060-00c00ce40d0e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:55.321001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401536a-5b4a-4682-b2d4-7fd93b1e60a5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:55.362823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf82e1e-9044-47e4-9019-1f252262962a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:55.363450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf82e1e-9044-47e4-9019-1f252262962a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:55.363730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf82e1e-9044-47e4-9019-1f252262962a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:55.364115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf82e1e-9044-47e4-9019-1f252262962a None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:55.364799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf82e1e-9044-47e4-9019-1f252262962a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:55.365101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf82e1e-9044-47e4-9019-1f252262962a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:55.366561 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf82e1e-9044-47e4-9019-1f252262962a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:55.381164 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 426/849] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:55.387384 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf82e1e-9044-47e4-9019-1f252262962a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:55.483342 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454d6f7-6209-4ffb-8060-00c00ce40d0e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:55.544516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:55.557613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a43ef-c3f3-4e56-bcec-55a4ab960b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:55.558949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a43ef-c3f3-4e56-bcec-55a4ab960b2d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:55.559547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a43ef-c3f3-4e56-bcec-55a4ab960b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:55.559982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a43ef-c3f3-4e56-bcec-55a4ab960b2d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:55.569131 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf82e1e-9044-47e4-9019-1f252262962a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:55.579185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a43ef-c3f3-4e56-bcec-55a4ab960b2d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:55.582092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a43ef-c3f3-4e56-bcec-55a4ab960b2d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:55.603491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-8d2a43ef-c3f3-4e56-bcec-55a4ab960b2d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:55.612007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:55.614310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:55.616844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:55.616844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:55.616844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:55.616844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:55.616844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:55.616844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:55.616844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:55.616844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:55.616844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 08:26:55.616844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:55.616844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 424/850] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:26:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:55.627042 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a43ef-c3f3-4e56-bcec-55a4ab960b2d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:55.644285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa5c0-2a9d-4167-aa25-e3cde4c66f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:55.644943 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa5c0-2a9d-4167-aa25-e3cde4c66f23 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:55.645222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa5c0-2a9d-4167-aa25-e3cde4c66f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:55.645703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa5c0-2a9d-4167-aa25-e3cde4c66f23 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:55.654536 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3aa5c0-2a9d-4167-aa25-e3cde4c66f23 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:55.656004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3aa5c0-2a9d-4167-aa25-e3cde4c66f23 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4c3aa5c0-2a9d-4167-aa25-e3cde4c66f23 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:26:55.671253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:55.678031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:55.679014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:55.679014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:55.679014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:26:55.679014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:26:55.679014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:26:55.679014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:26:55.679014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:26:55.679014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:26:55.679014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 08:26:55.679014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:55.679014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 427/851] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 08:26:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:55.683920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa5c0-2a9d-4167-aa25-e3cde4c66f23 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:55.759131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a43ef-c3f3-4e56-bcec-55a4ab960b2d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:55.802925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9bd67-b9d1-463f-a57d-0e6b44619b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:55.803705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9bd67-b9d1-463f-a57d-0e6b44619b82 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:55.804087 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9bd67-b9d1-463f-a57d-0e6b44619b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:55.804470 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9bd67-b9d1-463f-a57d-0e6b44619b82 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:55.805315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9bd67-b9d1-463f-a57d-0e6b44619b82 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:55.805558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa5c0-2a9d-4167-aa25-e3cde4c66f23 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:55.805839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9bd67-b9d1-463f-a57d-0e6b44619b82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:55.807500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9bd67-b9d1-463f-a57d-0e6b44619b82 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:55.836363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 425/852] 10.222.0.44 () {56 vars in 1135 bytes} [Sun Sep 11 08:26:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:55.838242 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9bd67-b9d1-463f-a57d-0e6b44619b82 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:55.877288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce6eac-2671-4376-8dde-594d153a8c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:55.879879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6eac-2671-4376-8dde-594d153a8c4b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:55.879879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6eac-2671-4376-8dde-594d153a8c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:55.879879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6eac-2671-4376-8dde-594d153a8c4b None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:55.880557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce6eac-2671-4376-8dde-594d153a8c4b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:55.880727 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce6eac-2671-4376-8dde-594d153a8c4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:55.882251 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce6eac-2671-4376-8dde-594d153a8c4b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:55.904721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 428/853] 10.222.0.44 () {56 vars in 1149 bytes} [Sun Sep 11 08:26:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:55.913481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce6eac-2671-4376-8dde-594d153a8c4b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:55.960820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:55.971164 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:56.039544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9bd67-b9d1-463f-a57d-0e6b44619b82 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:56.054458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce6eac-2671-4376-8dde-594d153a8c4b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:56.082269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4328a-255b-4faa-ab75-431988af8669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:56.082903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4328a-255b-4faa-ab75-431988af8669 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:56.083186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4328a-255b-4faa-ab75-431988af8669 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:56.083444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4328a-255b-4faa-ab75-431988af8669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:56.121285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2fe8a-b5b8-4613-9dd6-30038715fc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:56.122452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2fe8a-b5b8-4613-9dd6-30038715fc70 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:56.123095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2fe8a-b5b8-4613-9dd6-30038715fc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:56.123575 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2fe8a-b5b8-4613-9dd6-30038715fc70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:56.165753 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a4328a-255b-4faa-ab75-431988af8669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:56.167279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a4328a-255b-4faa-ab75-431988af8669 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:56.220751 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:56.256215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be2fe8a-b5b8-4613-9dd6-30038715fc70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:56.257675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be2fe8a-b5b8-4613-9dd6-30038715fc70 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:56.293122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 426/854] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:26:56.294833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56328f0-f9e8-44d7-b798-6fc7cf6ff94f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:56.295254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56328f0-f9e8-44d7-b798-6fc7cf6ff94f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:56.296011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56328f0-f9e8-44d7-b798-6fc7cf6ff94f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:56.298455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 427/855] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:56.299294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56328f0-f9e8-44d7-b798-6fc7cf6ff94f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:56.315759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:56.412713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 429/856] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:26:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 08:26:56.415475 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c89228-f81e-4ac2-bfb1-d6d572fe8cd9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:56.416095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c89228-f81e-4ac2-bfb1-d6d572fe8cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:56.416577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c89228-f81e-4ac2-bfb1-d6d572fe8cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:56.448816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e3c89228-f81e-4ac2-bfb1-d6d572fe8cd9 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:56.450763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e3c89228-f81e-4ac2-bfb1-d6d572fe8cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:56.498104 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56328f0-f9e8-44d7-b798-6fc7cf6ff94f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:56.563905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5ce5e-2d90-4ec1-bf75-0d26c8740730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:56.564854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5ce5e-2d90-4ec1-bf75-0d26c8740730 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:56.565332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5ce5e-2d90-4ec1-bf75-0d26c8740730 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:56.565803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5ce5e-2d90-4ec1-bf75-0d26c8740730 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:56.571232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 430/857] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:26:56.573183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c89228-f81e-4ac2-bfb1-d6d572fe8cd9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:56.585310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5ce5e-2d90-4ec1-bf75-0d26c8740730 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:56.588154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5ce5e-2d90-4ec1-bf75-0d26c8740730 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-96b5ce5e-2d90-4ec1-bf75-0d26c8740730 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:56.616739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:56.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:56.620280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:56.620280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:56.620280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:56.620280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 428/858] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:56.624770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5ce5e-2d90-4ec1-bf75-0d26c8740730 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:56.628488 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:56.733200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:56.771154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5ce5e-2d90-4ec1-bf75-0d26c8740730 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:56.786863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c89228-f81e-4ac2-bfb1-d6d572fe8cd9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:56.839515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b9847-5686-42cd-944a-79a478dcd5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:56.839951 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31cfaa-27ba-4633-84a1-de4ad8b0fe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:56.840828 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31cfaa-27ba-4633-84a1-de4ad8b0fe89 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:56.841119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31cfaa-27ba-4633-84a1-de4ad8b0fe89 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:56.841373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31cfaa-27ba-4633-84a1-de4ad8b0fe89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:56.842670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b9847-5686-42cd-944a-79a478dcd5fe None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:56.843751 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b9847-5686-42cd-944a-79a478dcd5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:56.845576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b9847-5686-42cd-944a-79a478dcd5fe None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:56.847169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b9847-5686-42cd-944a-79a478dcd5fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:56.848027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b9847-5686-42cd-944a-79a478dcd5fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:56.852204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b9847-5686-42cd-944a-79a478dcd5fe None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:56.859170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31cfaa-27ba-4633-84a1-de4ad8b0fe89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:56.859170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31cfaa-27ba-4633-84a1-de4ad8b0fe89 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-bd31cfaa-27ba-4633-84a1-de4ad8b0fe89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:26:56.884924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:56.886759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:26:56.888474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:26:56.888474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:26:56.888474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:26:56.891139 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 431/859] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:26:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:56.892121 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31cfaa-27ba-4633-84a1-de4ad8b0fe89 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:56.893815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 429/860] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:56.902622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b9847-5686-42cd-944a-79a478dcd5fe None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:57.017188 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31cfaa-27ba-4633-84a1-de4ad8b0fe89 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:57.018999 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:57.041705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:57.063266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607436c8-f15f-4e61-8e07-6a17dfe50afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:57.063878 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607436c8-f15f-4e61-8e07-6a17dfe50afd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:57.064155 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607436c8-f15f-4e61-8e07-6a17dfe50afd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:57.064425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607436c8-f15f-4e61-8e07-6a17dfe50afd None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:57.065081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607436c8-f15f-4e61-8e07-6a17dfe50afd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:57.065370 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607436c8-f15f-4e61-8e07-6a17dfe50afd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:57.066886 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607436c8-f15f-4e61-8e07-6a17dfe50afd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:57.081232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 432/861] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:57.082101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607436c8-f15f-4e61-8e07-6a17dfe50afd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:57.095377 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b9847-5686-42cd-944a-79a478dcd5fe None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:57.149287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b293986-a319-4600-b815-2ba710bb7f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:57.149924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b293986-a319-4600-b815-2ba710bb7f44 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:57.150241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b293986-a319-4600-b815-2ba710bb7f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:57.150945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b293986-a319-4600-b815-2ba710bb7f44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:57.160165 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b293986-a319-4600-b815-2ba710bb7f44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:57.162261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b293986-a319-4600-b815-2ba710bb7f44 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9b293986-a319-4600-b815-2ba710bb7f44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:57.177687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:57.179512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:57.182206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:57.182206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:57.182206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:57.182206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:57.182206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:57.182206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:57.182206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:57.182206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:57.182206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 430/862] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:57.182206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b293986-a319-4600-b815-2ba710bb7f44 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:57.231841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607436c8-f15f-4e61-8e07-6a17dfe50afd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:57.293768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8992f95-b934-4ef6-bbea-07fa9df9c622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:57.294770 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8992f95-b934-4ef6-bbea-07fa9df9c622 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:57.295219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8992f95-b934-4ef6-bbea-07fa9df9c622 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:57.295654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8992f95-b934-4ef6-bbea-07fa9df9c622 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:57.296770 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8992f95-b934-4ef6-bbea-07fa9df9c622 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:57.299890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8992f95-b934-4ef6-bbea-07fa9df9c622 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:57.334506 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b293986-a319-4600-b815-2ba710bb7f44 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:57.376475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:57.396811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 433/863] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:26:57 2022] POST /identity/v3/services => generated 251 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:26:57.398517 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8992f95-b934-4ef6-bbea-07fa9df9c622 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:57.411714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d7181-d127-4948-a59b-02b7a5a1178a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:57.416020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7181-d127-4948-a59b-02b7a5a1178a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:57.416020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7181-d127-4948-a59b-02b7a5a1178a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:57.416020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7181-d127-4948-a59b-02b7a5a1178a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:57.428075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d7181-d127-4948-a59b-02b7a5a1178a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:57.431019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d7181-d127-4948-a59b-02b7a5a1178a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2e8d7181-d127-4948-a59b-02b7a5a1178a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:26:57.457491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:57.459260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:57.461790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:26:57.461790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:26:57.461790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:26:57.461790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:26:57.461790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:26:57.461790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:26:57.461790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:26:57.461790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:57.466740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 431/864] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:26:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:26:57.471462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d7181-d127-4948-a59b-02b7a5a1178a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:57.517097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:57.647900 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8992f95-b934-4ef6-bbea-07fa9df9c622 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:57.708629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d7181-d127-4948-a59b-02b7a5a1178a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:57.724992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92571183-64d2-4fb8-b1a6-757d7ad6e974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:57.726486 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92571183-64d2-4fb8-b1a6-757d7ad6e974 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:57.727223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92571183-64d2-4fb8-b1a6-757d7ad6e974 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:57.728418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92571183-64d2-4fb8-b1a6-757d7ad6e974 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:57.729627 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92571183-64d2-4fb8-b1a6-757d7ad6e974 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:57.730565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92571183-64d2-4fb8-b1a6-757d7ad6e974 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:57.733134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92571183-64d2-4fb8-b1a6-757d7ad6e974 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:57.757316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 434/865] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:57.770992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92571183-64d2-4fb8-b1a6-757d7ad6e974 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:57.774623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238163cb-133c-4faa-a9fb-87783ae8ee56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:57.775565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238163cb-133c-4faa-a9fb-87783ae8ee56 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:57.776007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238163cb-133c-4faa-a9fb-87783ae8ee56 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:57.776450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238163cb-133c-4faa-a9fb-87783ae8ee56 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:57.777625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238163cb-133c-4faa-a9fb-87783ae8ee56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:57.778136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238163cb-133c-4faa-a9fb-87783ae8ee56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:57.780701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238163cb-133c-4faa-a9fb-87783ae8ee56 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:57.806140 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 432/866] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:26:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:57.816416 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238163cb-133c-4faa-a9fb-87783ae8ee56 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:57.913149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:57.976722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:57.990490 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92571183-64d2-4fb8-b1a6-757d7ad6e974 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:58.043528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238163cb-133c-4faa-a9fb-87783ae8ee56 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:58.067609 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5ac0f-a369-49d7-a273-35594f77dbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:58.068984 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5ac0f-a369-49d7-a273-35594f77dbad None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:58.069757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5ac0f-a369-49d7-a273-35594f77dbad None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:58.070376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5ac0f-a369-49d7-a273-35594f77dbad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:58.071775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5ac0f-a369-49d7-a273-35594f77dbad None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=e60a2198022c4f979a6e5e9057558c33) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:58.072356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5ac0f-a369-49d7-a273-35594f77dbad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:58.075321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5ac0f-a369-49d7-a273-35594f77dbad None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:58.085608 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 435/867] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:57 2022] GET /identity/v3/role_assignments?user.id=e60a2198022c4f979a6e5e9057558c33&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=020bc02dca71485bab2312442adaa179 => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:58.086946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1b67e-96b2-485f-9897-3a07c9830deb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:58.087214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1b67e-96b2-485f-9897-3a07c9830deb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:58.087490 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1b67e-96b2-485f-9897-3a07c9830deb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:58.093444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 436/868] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:26:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:26:58.094946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0d780-80f6-4dc9-a280-8c64ee0c7bb0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:58.095240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0d780-80f6-4dc9-a280-8c64ee0c7bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:58.095512 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0d780-80f6-4dc9-a280-8c64ee0c7bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:58.109514 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de5893-b8be-4f3b-8f47-fbf81c661e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:58.111044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de5893-b8be-4f3b-8f47-fbf81c661e9d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:58.111468 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de5893-b8be-4f3b-8f47-fbf81c661e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:58.111907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de5893-b8be-4f3b-8f47-fbf81c661e9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:58.113097 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de5893-b8be-4f3b-8f47-fbf81c661e9d None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=dab0506091034403bf2f0fe5f7538cd6) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:58.113571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de5893-b8be-4f3b-8f47-fbf81c661e9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:58.116030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de5893-b8be-4f3b-8f47-fbf81c661e9d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:58.116858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ede0d780-80f6-4dc9-a280-8c64ee0c7bb0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:26:58.118368 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ede0d780-80f6-4dc9-a280-8c64ee0c7bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:26:58.128750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 433/869] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:26:57 2022] GET /identity/v3/role_assignments?user.id=dab0506091034403bf2f0fe5f7538cd6&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:26:58.210199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 437/870] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:26:58 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:26:58.492550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de5893-b8be-4f3b-8f47-fbf81c661e9d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:58.626730 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de5893-b8be-4f3b-8f47-fbf81c661e9d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:58.628934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:58.669967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219d242-76ae-4165-a707-2c89a7b1b1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:58.671155 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219d242-76ae-4165-a707-2c89a7b1b1fd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:58.671644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219d242-76ae-4165-a707-2c89a7b1b1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:58.672650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219d242-76ae-4165-a707-2c89a7b1b1fd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:58.673483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219d242-76ae-4165-a707-2c89a7b1b1fd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:58.675166 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219d242-76ae-4165-a707-2c89a7b1b1fd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c219d242-76ae-4165-a707-2c89a7b1b1fd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:26:58.685807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:26:58.686928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 08:26:58.688441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:26:58.691585 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 434/871] 10.222.0.44 () {56 vars in 1132 bytes} [Sun Sep 11 08:26:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:26:58.702101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0d780-80f6-4dc9-a280-8c64ee0c7bb0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:58.890827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:26:58.903209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0d780-80f6-4dc9-a280-8c64ee0c7bb0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:58.945022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930007b-939a-4f8e-9829-f51837c92e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:58.945638 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930007b-939a-4f8e-9829-f51837c92e11 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:58.946032 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930007b-939a-4f8e-9829-f51837c92e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:58.946342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930007b-939a-4f8e-9829-f51837c92e11 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:58.946978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930007b-939a-4f8e-9829-f51837c92e11 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:58.947343 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930007b-939a-4f8e-9829-f51837c92e11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:58.948764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930007b-939a-4f8e-9829-f51837c92e11 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:58.954461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 438/872] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 08:26:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:26:58.959858 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219d242-76ae-4165-a707-2c89a7b1b1fd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:26:59.107654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219d242-76ae-4165-a707-2c89a7b1b1fd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:26:59.146469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54dd80-72d7-41e5-bd01-66cbcfc31b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:26:59.146993 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54dd80-72d7-41e5-bd01-66cbcfc31b9b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:26:59.147277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54dd80-72d7-41e5-bd01-66cbcfc31b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:26:59.147553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54dd80-72d7-41e5-bd01-66cbcfc31b9b None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:26:59.148209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54dd80-72d7-41e5-bd01-66cbcfc31b9b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:26:59.148513 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54dd80-72d7-41e5-bd01-66cbcfc31b9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:26:59.149947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54dd80-72d7-41e5-bd01-66cbcfc31b9b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:26:59.155740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 435/873] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 08:26:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:00.966390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e8051-566d-4413-afa5-4e82b9e25cd9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:00.967362 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e8051-566d-4413-afa5-4e82b9e25cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:00.967873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e8051-566d-4413-afa5-4e82b9e25cd9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:00.970128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 439/874] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:00.976199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cd2f6-5b7a-45f5-b705-154b503ca814 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:00.977109 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cd2f6-5b7a-45f5-b705-154b503ca814 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:00.977581 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cd2f6-5b7a-45f5-b705-154b503ca814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:00.997684 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b55cd2f6-5b7a-45f5-b705-154b503ca814 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:00.999323 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b55cd2f6-5b7a-45f5-b705-154b503ca814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:01.027191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:01.075378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 436/875] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:00 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:01.344919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e8051-566d-4413-afa5-4e82b9e25cd9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:01.392234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b38657-1689-4371-bbb2-947878cde7ce None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:01.393182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b38657-1689-4371-bbb2-947878cde7ce None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:01.393909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b38657-1689-4371-bbb2-947878cde7ce None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:01.397013 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 437/876] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:01.405975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3e40f-28ce-4dee-96e6-5ddaac6cab89 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:01.407414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3e40f-28ce-4dee-96e6-5ddaac6cab89 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:01.408204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3e40f-28ce-4dee-96e6-5ddaac6cab89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:01.439194 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-30b3e40f-28ce-4dee-96e6-5ddaac6cab89 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:01.441612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-30b3e40f-28ce-4dee-96e6-5ddaac6cab89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:01.460858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e8051-566d-4413-afa5-4e82b9e25cd9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:01.498136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c08743-81ce-46c4-893c-0f0c26466ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:01.499072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c08743-81ce-46c4-893c-0f0c26466ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:01.499351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c08743-81ce-46c4-893c-0f0c26466ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:01.499620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c08743-81ce-46c4-893c-0f0c26466ad7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:01.500230 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c08743-81ce-46c4-893c-0f0c26466ad7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:01.501592 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c08743-81ce-46c4-893c-0f0c26466ad7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-57c08743-81ce-46c4-893c-0f0c26466ad7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:01.511083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:01.512214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:01.513072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 440/877] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:27:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:01.517408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea64dee-dc42-44a1-8686-e530724dd595 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:01.517979 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea64dee-dc42-44a1-8686-e530724dd595 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:01.518454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea64dee-dc42-44a1-8686-e530724dd595 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:01.523989 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 441/878] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:01.523989 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea64dee-dc42-44a1-8686-e530724dd595 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:01.548887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:01.571101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 438/879] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:01.573272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8417d-7030-474e-b318-837de4a9b2d8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:01.573761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8417d-7030-474e-b318-837de4a9b2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:01.574353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8417d-7030-474e-b318-837de4a9b2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:01.612239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-47a8417d-7030-474e-b318-837de4a9b2d8 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:01.614281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-47a8417d-7030-474e-b318-837de4a9b2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:01.672032 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea64dee-dc42-44a1-8686-e530724dd595 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:01.702652 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 439/880] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:01.735965 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca25578-7843-416c-8b93-dffd5e0320ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:01.737064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca25578-7843-416c-8b93-dffd5e0320ec None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:01.737609 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca25578-7843-416c-8b93-dffd5e0320ec None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:01.738181 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca25578-7843-416c-8b93-dffd5e0320ec None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:01.739304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca25578-7843-416c-8b93-dffd5e0320ec None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:01.739845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca25578-7843-416c-8b93-dffd5e0320ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:01.742443 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca25578-7843-416c-8b93-dffd5e0320ec None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:01.752199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 442/881] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:27:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:01.757770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8417d-7030-474e-b318-837de4a9b2d8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:01.784593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:01.849366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f1fb5-d22f-4b71-b215-97c030281fe6 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:01.849784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f1fb5-d22f-4b71-b215-97c030281fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:01.850400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f1fb5-d22f-4b71-b215-97c030281fe6 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:01.858702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 443/882] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:01 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:27:01.870333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718417a-452b-497a-a2b6-d366f974e2aa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:01.870717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718417a-452b-497a-a2b6-d366f974e2aa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:01.871183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718417a-452b-497a-a2b6-d366f974e2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:01.872859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8417d-7030-474e-b318-837de4a9b2d8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:01.916780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646feaf7-07bc-4dfb-91e0-2f407aeb3582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:01.917058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8718417a-452b-497a-a2b6-d366f974e2aa None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:01.917444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646feaf7-07bc-4dfb-91e0-2f407aeb3582 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:01.917692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646feaf7-07bc-4dfb-91e0-2f407aeb3582 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:01.917949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646feaf7-07bc-4dfb-91e0-2f407aeb3582 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:01.918602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646feaf7-07bc-4dfb-91e0-2f407aeb3582 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:01.918899 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646feaf7-07bc-4dfb-91e0-2f407aeb3582 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:01.919054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8718417a-452b-497a-a2b6-d366f974e2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:01.920357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646feaf7-07bc-4dfb-91e0-2f407aeb3582 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:01.931796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 440/883] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:27:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:01.933347 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646feaf7-07bc-4dfb-91e0-2f407aeb3582 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:02.022427 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 444/884] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:02.022427 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718417a-452b-497a-a2b6-d366f974e2aa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:02.034706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:02.047583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:02.053651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646feaf7-07bc-4dfb-91e0-2f407aeb3582 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:02.094602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876e64b-45bd-4b40-bd88-b19b0f4ef601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:02.095210 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876e64b-45bd-4b40-bd88-b19b0f4ef601 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:02.095483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876e64b-45bd-4b40-bd88-b19b0f4ef601 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:02.095751 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876e64b-45bd-4b40-bd88-b19b0f4ef601 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:02.096519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876e64b-45bd-4b40-bd88-b19b0f4ef601 None admin] RBAC: Adding query filter params (interface=public, service_id=36973f6745134314a74650bdb17c5535, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:02.096878 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876e64b-45bd-4b40-bd88-b19b0f4ef601 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:02.098396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876e64b-45bd-4b40-bd88-b19b0f4ef601 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:02.109737 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 441/885] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 08:27:01 2022] GET /identity/v3/endpoints?service_id=36973f6745134314a74650bdb17c5535&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:02.111893 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876e64b-45bd-4b40-bd88-b19b0f4ef601 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:02.201275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718417a-452b-497a-a2b6-d366f974e2aa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:02.262001 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876e64b-45bd-4b40-bd88-b19b0f4ef601 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:02.264380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d57693-1df2-4418-b185-263337d83360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:02.265734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d57693-1df2-4418-b185-263337d83360 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:02.266249 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d57693-1df2-4418-b185-263337d83360 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:02.266733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d57693-1df2-4418-b185-263337d83360 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:02.267800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d57693-1df2-4418-b185-263337d83360 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:02.270925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d57693-1df2-4418-b185-263337d83360 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-84d57693-1df2-4418-b185-263337d83360 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:02.289714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:02.291838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:02.293418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 445/886] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 08:27:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:02.298030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d57693-1df2-4418-b185-263337d83360 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:02.309398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a82da6-eeb4-4fd2-a985-2991af0a4de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:02.310096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a82da6-eeb4-4fd2-a985-2991af0a4de0 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:02.310403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a82da6-eeb4-4fd2-a985-2991af0a4de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:02.310670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a82da6-eeb4-4fd2-a985-2991af0a4de0 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:02.311327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a82da6-eeb4-4fd2-a985-2991af0a4de0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:02.313300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a82da6-eeb4-4fd2-a985-2991af0a4de0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:02.421728 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 442/887] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:27:02 2022] POST /identity/v3/services => generated 269 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:27:02.423431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a82da6-eeb4-4fd2-a985-2991af0a4de0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:02.439883 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:02.445731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:02.487313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d57693-1df2-4418-b185-263337d83360 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:02.581224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb67da6-e64d-49db-8669-f63d894152a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:02.582115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb67da6-e64d-49db-8669-f63d894152a0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:02.582557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb67da6-e64d-49db-8669-f63d894152a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:02.582939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb67da6-e64d-49db-8669-f63d894152a0 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:02.583929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb67da6-e64d-49db-8669-f63d894152a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:02.586685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb67da6-e64d-49db-8669-f63d894152a0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:02.601435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a82da6-eeb4-4fd2-a985-2991af0a4de0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:02.606722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 446/888] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 08:27:02 2022] GET /identity/v3/services => generated 1268 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:27:02.608752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb67da6-e64d-49db-8669-f63d894152a0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:02.669234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62288b1-7a39-410f-93d2-4f659c073742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:02.670349 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62288b1-7a39-410f-93d2-4f659c073742 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:02.670654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62288b1-7a39-410f-93d2-4f659c073742 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:02.671101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62288b1-7a39-410f-93d2-4f659c073742 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:02.672749 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62288b1-7a39-410f-93d2-4f659c073742 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:02.673301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62288b1-7a39-410f-93d2-4f659c073742 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:02.675623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62288b1-7a39-410f-93d2-4f659c073742 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:02.687152 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 443/889] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 08:27:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:02.702200 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62288b1-7a39-410f-93d2-4f659c073742 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:02.811850 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb67da6-e64d-49db-8669-f63d894152a0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:02.869195 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:02.889495 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216f67a-ca1a-41b2-b78e-fe6dc9d7206f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:02.893385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216f67a-ca1a-41b2-b78e-fe6dc9d7206f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:02.893385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216f67a-ca1a-41b2-b78e-fe6dc9d7206f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:02.893385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216f67a-ca1a-41b2-b78e-fe6dc9d7206f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:02.900081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:02.916428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216f67a-ca1a-41b2-b78e-fe6dc9d7206f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6216f67a-ca1a-41b2-b78e-fe6dc9d7206f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:27:02.928562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:02.930523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:02.933190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:02.933190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:02.933190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:02.933190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:02.933190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:27:02.933190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:27:02.933190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:27:02.933190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:27:02.933190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:27:02.933190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:27:02.933190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 08:27:02.933190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:02.934686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62288b1-7a39-410f-93d2-4f659c073742 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:02.934860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 447/890] 10.222.0.44 () {56 vars in 1120 bytes} [Sun Sep 11 08:27:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:02.940212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216f67a-ca1a-41b2-b78e-fe6dc9d7206f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:02.996142 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d3fe5-a044-4d7a-98c5-87eef082be3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:02.997258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d3fe5-a044-4d7a-98c5-87eef082be3a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:02.997752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d3fe5-a044-4d7a-98c5-87eef082be3a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:02.998283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d3fe5-a044-4d7a-98c5-87eef082be3a None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:02.999455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d3fe5-a044-4d7a-98c5-87eef082be3a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:03.000005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d3fe5-a044-4d7a-98c5-87eef082be3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:03.002839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d3fe5-a044-4d7a-98c5-87eef082be3a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:03.011883 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 444/891] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 08:27:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:03.090516 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216f67a-ca1a-41b2-b78e-fe6dc9d7206f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:03.134362 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0f305-675e-4cda-95a3-3d1d660bbaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:03.134996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0f305-675e-4cda-95a3-3d1d660bbaff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:03.135272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0f305-675e-4cda-95a3-3d1d660bbaff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:03.135542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0f305-675e-4cda-95a3-3d1d660bbaff None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:03.136239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0f305-675e-4cda-95a3-3d1d660bbaff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:03.136541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0f305-675e-4cda-95a3-3d1d660bbaff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:03.137973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0f305-675e-4cda-95a3-3d1d660bbaff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:03.150707 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 448/892] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 08:27:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:03.160552 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d3fe5-a044-4d7a-98c5-87eef082be3a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:03.351811 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d3fe5-a044-4d7a-98c5-87eef082be3a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:03.408580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed8f3c-812d-4565-9f55-9ec5da2b87fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:03.409224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed8f3c-812d-4565-9f55-9ec5da2b87fe None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:03.409498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed8f3c-812d-4565-9f55-9ec5da2b87fe None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:03.409770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed8f3c-812d-4565-9f55-9ec5da2b87fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:03.412870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:03.422365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed8f3c-812d-4565-9f55-9ec5da2b87fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:03.423921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed8f3c-812d-4565-9f55-9ec5da2b87fe None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-31ed8f3c-812d-4565-9f55-9ec5da2b87fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:27:03.440190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:03.442330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:27:03.444249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:27:03.444249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:03.444249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:03.444249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 445/893] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:03.455642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0f305-675e-4cda-95a3-3d1d660bbaff None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:03.528441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:03.578906 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0f305-675e-4cda-95a3-3d1d660bbaff None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:03.618702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c4061-83b3-46b8-8d8e-e03f6d9191cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:03.619222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c4061-83b3-46b8-8d8e-e03f6d9191cc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:03.619501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c4061-83b3-46b8-8d8e-e03f6d9191cc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:03.619773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c4061-83b3-46b8-8d8e-e03f6d9191cc None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:03.620408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c4061-83b3-46b8-8d8e-e03f6d9191cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:03.620706 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c4061-83b3-46b8-8d8e-e03f6d9191cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:03.622088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c4061-83b3-46b8-8d8e-e03f6d9191cc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:03.636186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 449/894] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:03.646304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed8f3c-812d-4565-9f55-9ec5da2b87fe None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:03.810754 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed8f3c-812d-4565-9f55-9ec5da2b87fe None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:03.848910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9439e-431b-4d0f-95bc-63bb60f80eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:03.849912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9439e-431b-4d0f-95bc-63bb60f80eeb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:03.850443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9439e-431b-4d0f-95bc-63bb60f80eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:03.850912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9439e-431b-4d0f-95bc-63bb60f80eeb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:03.863443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9439e-431b-4d0f-95bc-63bb60f80eeb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:03.865258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9439e-431b-4d0f-95bc-63bb60f80eeb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b9a9439e-431b-4d0f-95bc-63bb60f80eeb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:27:03.881778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:03.882787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:03.883732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:03.883732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:03.883732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:27:03.883732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:27:03.883732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:27:03.883732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:27:03.883732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:27:03.883732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:27:03.883732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 08:27:03.883732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:03.887922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 446/895] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 08:27:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:03.889330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c4061-83b3-46b8-8d8e-e03f6d9191cc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:04.034189 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c4061-83b3-46b8-8d8e-e03f6d9191cc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:04.059113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:04.077298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf72476-ef29-4abe-a699-4712b11bc30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:04.077924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf72476-ef29-4abe-a699-4712b11bc30a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:04.078263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf72476-ef29-4abe-a699-4712b11bc30a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:04.078537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf72476-ef29-4abe-a699-4712b11bc30a None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:04.079280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf72476-ef29-4abe-a699-4712b11bc30a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:04.079587 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf72476-ef29-4abe-a699-4712b11bc30a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:04.081053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf72476-ef29-4abe-a699-4712b11bc30a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:04.102393 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 450/896] 10.222.0.44 () {56 vars in 1149 bytes} [Sun Sep 11 08:27:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:04.111733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9439e-431b-4d0f-95bc-63bb60f80eeb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:04.206089 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:04.271578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9439e-431b-4d0f-95bc-63bb60f80eeb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:04.309282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbd931-8c28-4205-8e0e-cc0fcd3b0598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:04.310255 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbd931-8c28-4205-8e0e-cc0fcd3b0598 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:04.310711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbd931-8c28-4205-8e0e-cc0fcd3b0598 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:04.311152 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbd931-8c28-4205-8e0e-cc0fcd3b0598 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:04.389189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbd931-8c28-4205-8e0e-cc0fcd3b0598 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:04.390773 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbd931-8c28-4205-8e0e-cc0fcd3b0598 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:04.449149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:04.512353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 447/897] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:27:04 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:27:04.522025 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf72476-ef29-4abe-a699-4712b11bc30a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:04.708553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf72476-ef29-4abe-a699-4712b11bc30a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:04.767431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c1211-f0c0-4973-a002-a78a9d5401fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:04.768085 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c1211-f0c0-4973-a002-a78a9d5401fc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:04.768365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c1211-f0c0-4973-a002-a78a9d5401fc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:04.768692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c1211-f0c0-4973-a002-a78a9d5401fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:04.781374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c1211-f0c0-4973-a002-a78a9d5401fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:04.783414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c1211-f0c0-4973-a002-a78a9d5401fc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f85c1211-f0c0-4973-a002-a78a9d5401fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:27:04.800288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:04.801315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:27:04.802307 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:27:04.802307 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:04.802307 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:04.802307 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 451/898] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:04.811034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbd931-8c28-4205-8e0e-cc0fcd3b0598 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:04.952769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbd931-8c28-4205-8e0e-cc0fcd3b0598 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:04.990561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d027566-c9e7-49e9-84ec-4f178824e36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:04.991176 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d027566-c9e7-49e9-84ec-4f178824e36d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:04.991448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d027566-c9e7-49e9-84ec-4f178824e36d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:04.991714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d027566-c9e7-49e9-84ec-4f178824e36d None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:04.992357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d027566-c9e7-49e9-84ec-4f178824e36d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:04.992654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d027566-c9e7-49e9-84ec-4f178824e36d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:04.994020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d027566-c9e7-49e9-84ec-4f178824e36d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:05.007833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 448/899] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:05.014086 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c1211-f0c0-4973-a002-a78a9d5401fc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:05.035422 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818f3ca-92be-4875-b4fd-07521607d54f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:05.035601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818f3ca-92be-4875-b4fd-07521607d54f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:05.035945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818f3ca-92be-4875-b4fd-07521607d54f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:05.037648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 449/900] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:05.043512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e8afa-3135-4b61-876f-6dd75d41206d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:05.043844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e8afa-3135-4b61-876f-6dd75d41206d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:05.044128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e8afa-3135-4b61-876f-6dd75d41206d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:05.053642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:05.066521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b40e8afa-3135-4b61-876f-6dd75d41206d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:05.078773 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b40e8afa-3135-4b61-876f-6dd75d41206d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:05.078773 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:05.145832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c1211-f0c0-4973-a002-a78a9d5401fc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:05.163079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 450/901] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:27:05.191397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62950901-04e4-4023-b180-a22b2ea07438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:05.192520 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62950901-04e4-4023-b180-a22b2ea07438 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:05.192992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62950901-04e4-4023-b180-a22b2ea07438 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:05.193455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62950901-04e4-4023-b180-a22b2ea07438 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:05.201456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62950901-04e4-4023-b180-a22b2ea07438 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:05.203921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62950901-04e4-4023-b180-a22b2ea07438 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-62950901-04e4-4023-b180-a22b2ea07438 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:27:05.217202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:05.218224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:05.219713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:27:05.219713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:27:05.219713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:27:05.219713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:27:05.219713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:27:05.219713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:27:05.219713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:05.219713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:05.226411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 452/902] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:27:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:05.228819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e8afa-3135-4b61-876f-6dd75d41206d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:05.347663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e8afa-3135-4b61-876f-6dd75d41206d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:05.385653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec43cb-a4ca-48de-894d-134a1e131c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:05.386680 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec43cb-a4ca-48de-894d-134a1e131c70 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:05.387178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec43cb-a4ca-48de-894d-134a1e131c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:05.387640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec43cb-a4ca-48de-894d-134a1e131c70 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:05.388541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec43cb-a4ca-48de-894d-134a1e131c70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:05.389050 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec43cb-a4ca-48de-894d-134a1e131c70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:05.391332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec43cb-a4ca-48de-894d-134a1e131c70 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:05.399553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:05.408139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 451/903] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:27:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:05.416198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62950901-04e4-4023-b180-a22b2ea07438 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:05.473874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec43cb-a4ca-48de-894d-134a1e131c70 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:05.527137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:05.543473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62950901-04e4-4023-b180-a22b2ea07438 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:05.590009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec43cb-a4ca-48de-894d-134a1e131c70 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:05.600946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd77333-2f47-485f-a1d4-0f66882fd58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:05.602680 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd77333-2f47-485f-a1d4-0f66882fd58a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:05.603640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd77333-2f47-485f-a1d4-0f66882fd58a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:05.604401 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd77333-2f47-485f-a1d4-0f66882fd58a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:05.606307 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd77333-2f47-485f-a1d4-0f66882fd58a None admin] RBAC: Adding query filter params (role.id=c856bb63a85742ce8066968715c33568, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=dab0506091034403bf2f0fe5f7538cd6) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:05.607372 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd77333-2f47-485f-a1d4-0f66882fd58a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:05.610717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd77333-2f47-485f-a1d4-0f66882fd58a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:05.628660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 453/904] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:27:05 2022] GET /identity/v3/role_assignments?user.id=dab0506091034403bf2f0fe5f7538cd6&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=c856bb63a85742ce8066968715c33568 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:05.632937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d848f3-8578-474a-a546-778953a57311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:05.633594 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d848f3-8578-474a-a546-778953a57311 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:05.633921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d848f3-8578-474a-a546-778953a57311 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:05.634185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d848f3-8578-474a-a546-778953a57311 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:05.634759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d848f3-8578-474a-a546-778953a57311 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:05.636179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d848f3-8578-474a-a546-778953a57311 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e6d848f3-8578-474a-a546-778953a57311 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:05.646716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:05.648925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:05.650982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 452/905] 10.222.0.44 () {56 vars in 1132 bytes} [Sun Sep 11 08:27:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:05.658424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd77333-2f47-485f-a1d4-0f66882fd58a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:05.731449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9a5a6-94c7-4adc-863e-0149ea787f0d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:05.731779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9a5a6-94c7-4adc-863e-0149ea787f0d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:05.732143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9a5a6-94c7-4adc-863e-0149ea787f0d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:05.733725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 453/906] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:05.739855 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca05d9-98b0-410d-87e2-a6cee8f5493d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:05.740169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca05d9-98b0-410d-87e2-a6cee8f5493d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:05.740309 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca05d9-98b0-410d-87e2-a6cee8f5493d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:05.761128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-68ca05d9-98b0-410d-87e2-a6cee8f5493d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:05.762385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-68ca05d9-98b0-410d-87e2-a6cee8f5493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:05.835849 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:05.862262 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd77333-2f47-485f-a1d4-0f66882fd58a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:05.868582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 454/907] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:05.913900 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76244e01-7ebf-4d27-877c-07381d0bb52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:05.914552 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76244e01-7ebf-4d27-877c-07381d0bb52a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:05.914844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76244e01-7ebf-4d27-877c-07381d0bb52a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:05.915209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76244e01-7ebf-4d27-877c-07381d0bb52a None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:05.915719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76244e01-7ebf-4d27-877c-07381d0bb52a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:05.915992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76244e01-7ebf-4d27-877c-07381d0bb52a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:05.917460 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76244e01-7ebf-4d27-877c-07381d0bb52a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:05.923845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 454/908] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 08:27:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:05.930372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca05d9-98b0-410d-87e2-a6cee8f5493d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:05.951043 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449b369-d3e5-4c74-bfe3-7de252c61129 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:05.951302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449b369-d3e5-4c74-bfe3-7de252c61129 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:05.951638 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449b369-d3e5-4c74-bfe3-7de252c61129 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:05.953807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 455/909] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:27:05.960237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282afb49-686b-44b5-ac44-86dd58c64896 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:05.960660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282afb49-686b-44b5-ac44-86dd58c64896 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:05.960812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282afb49-686b-44b5-ac44-86dd58c64896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:05.987992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-282afb49-686b-44b5-ac44-86dd58c64896 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:05.990615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-282afb49-686b-44b5-ac44-86dd58c64896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:06.013455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:06.100478 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca05d9-98b0-410d-87e2-a6cee8f5493d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:06.115119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 456/910] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:27:06.147569 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282afb49-686b-44b5-ac44-86dd58c64896 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:06.163181 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade5f30-affd-4b43-a6d6-27e78a17cc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:06.164217 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade5f30-affd-4b43-a6d6-27e78a17cc7d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:06.164558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade5f30-affd-4b43-a6d6-27e78a17cc7d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:06.164837 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade5f30-affd-4b43-a6d6-27e78a17cc7d None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:06.165562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade5f30-affd-4b43-a6d6-27e78a17cc7d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:06.165861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade5f30-affd-4b43-a6d6-27e78a17cc7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:06.167298 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade5f30-affd-4b43-a6d6-27e78a17cc7d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:06.173018 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 455/911] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 08:27:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:06.178545 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade5f30-affd-4b43-a6d6-27e78a17cc7d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:06.288282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:06.347073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade5f30-affd-4b43-a6d6-27e78a17cc7d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:06.348387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282afb49-686b-44b5-ac44-86dd58c64896 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:06.393750 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9e65f-d225-4fbb-ad28-0f69ebbe607e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:06.394505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e65f-d225-4fbb-ad28-0f69ebbe607e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:06.394883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e65f-d225-4fbb-ad28-0f69ebbe607e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:06.395247 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e65f-d225-4fbb-ad28-0f69ebbe607e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:06.395952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9e65f-d225-4fbb-ad28-0f69ebbe607e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:06.397849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9e65f-d225-4fbb-ad28-0f69ebbe607e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e3f9e65f-d225-4fbb-ad28-0f69ebbe607e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:06.409408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:06.411653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:06.413551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 457/912] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:27:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:06.419165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9e65f-d225-4fbb-ad28-0f69ebbe607e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:06.420796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178f511-218d-4746-99c3-e652041de747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:06.421417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178f511-218d-4746-99c3-e652041de747 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:06.421764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178f511-218d-4746-99c3-e652041de747 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:06.422075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178f511-218d-4746-99c3-e652041de747 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:06.422824 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178f511-218d-4746-99c3-e652041de747 None admin] RBAC: Adding query filter params (interface=public, service_id=2808326090d843fdb1e510a9c95c6174, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:06.423183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178f511-218d-4746-99c3-e652041de747 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:06.424539 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1178f511-218d-4746-99c3-e652041de747 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:06.431800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 456/913] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 08:27:06 2022] GET /identity/v3/endpoints?service_id=2808326090d843fdb1e510a9c95c6174&interface=public®ion_id=RegionOne => generated 194 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:06.433107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178f511-218d-4746-99c3-e652041de747 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:06.446118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:06.561402 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9e65f-d225-4fbb-ad28-0f69ebbe607e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:06.596857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178f511-218d-4746-99c3-e652041de747 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:06.604244 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289f141-ce3c-4bfd-9d70-80a813643046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:06.605140 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f141-ce3c-4bfd-9d70-80a813643046 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:06.605558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f141-ce3c-4bfd-9d70-80a813643046 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:06.605962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f141-ce3c-4bfd-9d70-80a813643046 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:06.607004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289f141-ce3c-4bfd-9d70-80a813643046 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:06.607482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289f141-ce3c-4bfd-9d70-80a813643046 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:06.610063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289f141-ce3c-4bfd-9d70-80a813643046 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:06.622733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 458/914] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:27:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:06.622733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289f141-ce3c-4bfd-9d70-80a813643046 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:06.644434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:06.664328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd8b1f-a901-45ba-bf1c-9ab02311db02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:06.664970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd8b1f-a901-45ba-bf1c-9ab02311db02 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:06.665245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd8b1f-a901-45ba-bf1c-9ab02311db02 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:06.665520 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd8b1f-a901-45ba-bf1c-9ab02311db02 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:06.666239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd8b1f-a901-45ba-bf1c-9ab02311db02 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:06.667695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd8b1f-a901-45ba-bf1c-9ab02311db02 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:06.743299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 457/915] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:27:06 2022] POST /identity/v3/services => generated 241 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:27:06.752543 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd8b1f-a901-45ba-bf1c-9ab02311db02 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:06.759375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:06.817444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289f141-ce3c-4bfd-9d70-80a813643046 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:06.924766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0ce48-6641-42ff-8b2e-d8bf3ccd63b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:06.926213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0ce48-6641-42ff-8b2e-d8bf3ccd63b9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:06.926930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0ce48-6641-42ff-8b2e-d8bf3ccd63b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:06.927713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0ce48-6641-42ff-8b2e-d8bf3ccd63b9 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:06.929331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0ce48-6641-42ff-8b2e-d8bf3ccd63b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:06.933325 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0ce48-6641-42ff-8b2e-d8bf3ccd63b9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:06.945848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 459/916] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 08:27:06 2022] GET /identity/v3/services => generated 1497 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:27:07.008504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd8b1f-a901-45ba-bf1c-9ab02311db02 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:07.078428 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8743578-eb62-4e72-8588-b03bbc30f045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:07.079604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8743578-eb62-4e72-8588-b03bbc30f045 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:07.080219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8743578-eb62-4e72-8588-b03bbc30f045 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:07.080873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8743578-eb62-4e72-8588-b03bbc30f045 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:07.081933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8743578-eb62-4e72-8588-b03bbc30f045 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:07.082635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8743578-eb62-4e72-8588-b03bbc30f045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:07.085382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8743578-eb62-4e72-8588-b03bbc30f045 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:07.096937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 458/917] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 08:27:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:07.106603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0ce48-6641-42ff-8b2e-d8bf3ccd63b9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:07.247154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0ce48-6641-42ff-8b2e-d8bf3ccd63b9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:07.284882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b9e19-bf85-46be-9b9b-f1c6c1cc44e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:07.285501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b9e19-bf85-46be-9b9b-f1c6c1cc44e4 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:07.285779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b9e19-bf85-46be-9b9b-f1c6c1cc44e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:07.286093 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b9e19-bf85-46be-9b9b-f1c6c1cc44e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:07.286770 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b9e19-bf85-46be-9b9b-f1c6c1cc44e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:07.288172 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b9e19-bf85-46be-9b9b-f1c6c1cc44e4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:07.293390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:07.392513 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 460/918] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:27:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:27:08.010193 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0796b-da3b-4ed0-acaa-4c36bc3adf6a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:08.010734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0796b-da3b-4ed0-acaa-4c36bc3adf6a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:08.011175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0796b-da3b-4ed0-acaa-4c36bc3adf6a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:08.013458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 459/919] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:08.020826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedbfce-f55d-43a0-8bcc-54e4873056c4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:08.021329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedbfce-f55d-43a0-8bcc-54e4873056c4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:08.021789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedbfce-f55d-43a0-8bcc-54e4873056c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:08.052599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1cedbfce-f55d-43a0-8bcc-54e4873056c4 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:08.054792 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1cedbfce-f55d-43a0-8bcc-54e4873056c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:08.195735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 461/920] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:08.466423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0796b-da3b-4ed0-acaa-4c36bc3adf6a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:08.523658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:08.620473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0796b-da3b-4ed0-acaa-4c36bc3adf6a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:08.684431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c50f5a-d73e-4e02-a4bf-ae921b8e6279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:08.685788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c50f5a-d73e-4e02-a4bf-ae921b8e6279 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:08.686552 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c50f5a-d73e-4e02-a4bf-ae921b8e6279 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:08.687240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c50f5a-d73e-4e02-a4bf-ae921b8e6279 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:08.707378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c50f5a-d73e-4e02-a4bf-ae921b8e6279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-75c50f5a-d73e-4e02-a4bf-ae921b8e6279 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:27:08.718635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:08.721139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:08.723108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:08.723108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:08.723108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:08.723108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:08.723108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:27:08.723108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:27:08.723108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:27:08.723108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:27:08.723108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:27:08.723108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:27:08.723108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 08:27:08.723108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:08.727085 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 460/921] 10.222.0.44 () {56 vars in 1120 bytes} [Sun Sep 11 08:27:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:08.735174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedbfce-f55d-43a0-8bcc-54e4873056c4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:08.859202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedbfce-f55d-43a0-8bcc-54e4873056c4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:08.897860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:08.933732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645bcaa0-a230-4c20-8995-9dff65e68a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:08.934834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bcaa0-a230-4c20-8995-9dff65e68a3a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:08.935400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bcaa0-a230-4c20-8995-9dff65e68a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:08.935881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bcaa0-a230-4c20-8995-9dff65e68a3a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:08.937144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645bcaa0-a230-4c20-8995-9dff65e68a3a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:08.937722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645bcaa0-a230-4c20-8995-9dff65e68a3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:08.940424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645bcaa0-a230-4c20-8995-9dff65e68a3a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:08.958288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 462/922] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 08:27:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:08.964816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c50f5a-d73e-4e02-a4bf-ae921b8e6279 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:09.089549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c50f5a-d73e-4e02-a4bf-ae921b8e6279 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:09.092445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:09.128263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec4b29-26cf-4f32-af39-8959fe94fe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:09.128905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec4b29-26cf-4f32-af39-8959fe94fe1c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:09.129237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec4b29-26cf-4f32-af39-8959fe94fe1c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:09.129489 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec4b29-26cf-4f32-af39-8959fe94fe1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:09.143293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec4b29-26cf-4f32-af39-8959fe94fe1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:09.144748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec4b29-26cf-4f32-af39-8959fe94fe1c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-96ec4b29-26cf-4f32-af39-8959fe94fe1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:27:09.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:09.165007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:27:09.170584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:27:09.170584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:09.170584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:09.170584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 461/923] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:09.170819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645bcaa0-a230-4c20-8995-9dff65e68a3a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:09.319033 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645bcaa0-a230-4c20-8995-9dff65e68a3a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:09.384881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b0513-d6c9-4ba3-bbfd-2b3a14c9bbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:09.385839 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b0513-d6c9-4ba3-bbfd-2b3a14c9bbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:09.386366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b0513-d6c9-4ba3-bbfd-2b3a14c9bbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:09.386841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b0513-d6c9-4ba3-bbfd-2b3a14c9bbb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:09.388028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b0513-d6c9-4ba3-bbfd-2b3a14c9bbb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:09.388568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b0513-d6c9-4ba3-bbfd-2b3a14c9bbb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:09.391796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b0513-d6c9-4ba3-bbfd-2b3a14c9bbb0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:09.415662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 463/924] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:09.421394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec4b29-26cf-4f32-af39-8959fe94fe1c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:09.540430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec4b29-26cf-4f32-af39-8959fe94fe1c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:09.580625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04d6cd-ad32-4b56-b519-164a0fc30173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:09.581675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04d6cd-ad32-4b56-b519-164a0fc30173 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:09.582180 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04d6cd-ad32-4b56-b519-164a0fc30173 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:09.582717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04d6cd-ad32-4b56-b519-164a0fc30173 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:09.591984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04d6cd-ad32-4b56-b519-164a0fc30173 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:09.593824 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04d6cd-ad32-4b56-b519-164a0fc30173 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ab04d6cd-ad32-4b56-b519-164a0fc30173 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:27:09.606819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:09.607844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:09.608789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:09.608789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:09.608789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:27:09.608789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:27:09.608789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:27:09.608789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:27:09.608789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:27:09.608789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:27:09.608789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 08:27:09.608789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:09.611618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 462/925] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 08:27:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:09.618472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b0513-d6c9-4ba3-bbfd-2b3a14c9bbb0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:09.702246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:09.779797 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885ca74-6aa5-4a1c-ab43-8e288c21fb65 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:09.780156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885ca74-6aa5-4a1c-ab43-8e288c21fb65 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:09.780325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885ca74-6aa5-4a1c-ab43-8e288c21fb65 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:09.783393 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 463/926] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:09.788094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3db77-7a59-4795-be4a-22d5ec590ce0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:09.788420 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3db77-7a59-4795-be4a-22d5ec590ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:09.788641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3db77-7a59-4795-be4a-22d5ec590ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:09.808924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-80b3db77-7a59-4795-be4a-22d5ec590ce0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:09.810186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-80b3db77-7a59-4795-be4a-22d5ec590ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:09.824677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b0513-d6c9-4ba3-bbfd-2b3a14c9bbb0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:09.827741 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:09.904284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8fc95-e400-4e45-9a31-f1bc29b6d1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:09.905366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8fc95-e400-4e45-9a31-f1bc29b6d1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:09.905829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8fc95-e400-4e45-9a31-f1bc29b6d1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:09.906380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8fc95-e400-4e45-9a31-f1bc29b6d1f5 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:09.907862 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8fc95-e400-4e45-9a31-f1bc29b6d1f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:09.908401 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8fc95-e400-4e45-9a31-f1bc29b6d1f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:09.911204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 464/927] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:27:09.911415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8fc95-e400-4e45-9a31-f1bc29b6d1f5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:09.949437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 464/928] 10.222.0.44 () {56 vars in 1149 bytes} [Sun Sep 11 08:27:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:09.955705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3db77-7a59-4795-be4a-22d5ec590ce0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:10.080739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3db77-7a59-4795-be4a-22d5ec590ce0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:10.121224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be06b31-39b9-4472-9b80-ffea5b6539d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:10.122738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be06b31-39b9-4472-9b80-ffea5b6539d3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:10.123221 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be06b31-39b9-4472-9b80-ffea5b6539d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:10.123712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be06b31-39b9-4472-9b80-ffea5b6539d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:10.135874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be06b31-39b9-4472-9b80-ffea5b6539d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:10.137865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be06b31-39b9-4472-9b80-ffea5b6539d3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:10.147343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3be06b31-39b9-4472-9b80-ffea5b6539d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:27:10.157911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:10.158994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:27:10.159894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:27:10.159894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:10.159894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:10.162139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 465/929] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:10.168972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8fc95-e400-4e45-9a31-f1bc29b6d1f5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:10.190955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be06b31-39b9-4472-9b80-ffea5b6539d3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:10.283516 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:10.290419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8fc95-e400-4e45-9a31-f1bc29b6d1f5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:10.332103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183e334-cb07-441d-ac31-ce2b0b4301da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:10.332710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183e334-cb07-441d-ac31-ce2b0b4301da None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:10.333003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183e334-cb07-441d-ac31-ce2b0b4301da None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:10.333274 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183e334-cb07-441d-ac31-ce2b0b4301da None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:10.335106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183e334-cb07-441d-ac31-ce2b0b4301da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:10.335106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183e334-cb07-441d-ac31-ce2b0b4301da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:10.335565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183e334-cb07-441d-ac31-ce2b0b4301da None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:10.356760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 465/930] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:10.356760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33dace-d7c5-44a8-aaed-e9624e13b1f9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:10.356760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33dace-d7c5-44a8-aaed-e9624e13b1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:10.356760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33dace-d7c5-44a8-aaed-e9624e13b1f9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:10.356760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 466/931] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:10.359183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33dace-d7c5-44a8-aaed-e9624e13b1f9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:10.395648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be06b31-39b9-4472-9b80-ffea5b6539d3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:10.437919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e69efa-01f3-4f93-9c8f-7577bcbe51ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:10.438700 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e69efa-01f3-4f93-9c8f-7577bcbe51ac None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:10.438970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e69efa-01f3-4f93-9c8f-7577bcbe51ac None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:10.439241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e69efa-01f3-4f93-9c8f-7577bcbe51ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:10.439833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e69efa-01f3-4f93-9c8f-7577bcbe51ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:10.441417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e69efa-01f3-4f93-9c8f-7577bcbe51ac None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-07e69efa-01f3-4f93-9c8f-7577bcbe51ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:10.451723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:10.454131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:10.456561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 466/932] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 08:27:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:10.458105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a61f9-3455-4086-bfcf-5d849769c37c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:10.458105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a61f9-3455-4086-bfcf-5d849769c37c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:10.458105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a61f9-3455-4086-bfcf-5d849769c37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:10.477998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4d6a61f9-3455-4086-bfcf-5d849769c37c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:10.480203 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4d6a61f9-3455-4086-bfcf-5d849769c37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:10.537376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33dace-d7c5-44a8-aaed-e9624e13b1f9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:10.574093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:10.600985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 467/933] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:10.601301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb313b8-152c-4884-9331-c3fca330f2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:10.601301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb313b8-152c-4884-9331-c3fca330f2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:10.601301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb313b8-152c-4884-9331-c3fca330f2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:10.601301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb313b8-152c-4884-9331-c3fca330f2a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:10.601880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a61f9-3455-4086-bfcf-5d849769c37c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:10.611397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb313b8-152c-4884-9331-c3fca330f2a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:10.613673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb313b8-152c-4884-9331-c3fca330f2a9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4bb313b8-152c-4884-9331-c3fca330f2a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:27:10.628768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:10.630729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:10.632435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:27:10.632435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:27:10.632435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:27:10.632435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:27:10.632435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:27:10.632435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:27:10.632435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:10.632435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:10.634099 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 467/934] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:27:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:10.641461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb313b8-152c-4884-9331-c3fca330f2a9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:10.680472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:10.755479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a61f9-3455-4086-bfcf-5d849769c37c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:10.817522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ce56c-1591-4301-a5f3-d2589091aa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:10.818618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ce56c-1591-4301-a5f3-d2589091aa1e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:10.819091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ce56c-1591-4301-a5f3-d2589091aa1e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:10.819544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ce56c-1591-4301-a5f3-d2589091aa1e None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:10.820590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ce56c-1591-4301-a5f3-d2589091aa1e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:10.821086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ce56c-1591-4301-a5f3-d2589091aa1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:10.823553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ce56c-1591-4301-a5f3-d2589091aa1e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:10.830853 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb313b8-152c-4884-9331-c3fca330f2a9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:10.833021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 468/935] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 08:27:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:10.835304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d0084-846d-476f-9b93-ac570c031ea7 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:10.835791 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d0084-846d-476f-9b93-ac570c031ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:10.836253 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d0084-846d-476f-9b93-ac570c031ea7 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:10.841276 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 469/936] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:10.842172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d0084-846d-476f-9b93-ac570c031ea7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:10.910821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6332d-7905-490e-a978-21eec0d3e607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:10.911813 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6332d-7905-490e-a978-21eec0d3e607 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:10.912304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6332d-7905-490e-a978-21eec0d3e607 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:10.912708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6332d-7905-490e-a978-21eec0d3e607 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:10.913532 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6332d-7905-490e-a978-21eec0d3e607 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:10.914086 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6332d-7905-490e-a978-21eec0d3e607 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:10.916872 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6332d-7905-490e-a978-21eec0d3e607 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:10.928598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:10.945050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 468/937] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:27:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:10.945050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6924a-adc9-4cd1-9960-3f5c12e39cef None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:10.945050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6924a-adc9-4cd1-9960-3f5c12e39cef None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:10.945050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6924a-adc9-4cd1-9960-3f5c12e39cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:10.973819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7cd6924a-adc9-4cd1-9960-3f5c12e39cef None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:10.975916 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7cd6924a-adc9-4cd1-9960-3f5c12e39cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:10.989057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d0084-846d-476f-9b93-ac570c031ea7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:11.025287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43851dd7-a315-47f0-a098-758bbb33041a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:11.025892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43851dd7-a315-47f0-a098-758bbb33041a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:11.026198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43851dd7-a315-47f0-a098-758bbb33041a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:11.026464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43851dd7-a315-47f0-a098-758bbb33041a None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:11.027085 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43851dd7-a315-47f0-a098-758bbb33041a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:11.027383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43851dd7-a315-47f0-a098-758bbb33041a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:11.028787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43851dd7-a315-47f0-a098-758bbb33041a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:11.033895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 470/938] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 08:27:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:11.034732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43851dd7-a315-47f0-a098-758bbb33041a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:11.102356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 469/939] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:11.102356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6924a-adc9-4cd1-9960-3f5c12e39cef None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:11.108313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:11.188892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43851dd7-a315-47f0-a098-758bbb33041a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:11.238036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:11.244313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6924a-adc9-4cd1-9960-3f5c12e39cef None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:11.267036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff947c-915a-4aed-8a85-65bd11886c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:11.268307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff947c-915a-4aed-8a85-65bd11886c5b None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:11.268774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff947c-915a-4aed-8a85-65bd11886c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:11.269246 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff947c-915a-4aed-8a85-65bd11886c5b None admin] PATH_INFO: `/v3/projects/811863e9c94d44a39647800f306505f0/users/dab0506091034403bf2f0fe5f7538cd6/roles/c856bb63a85742ce8066968715c33568` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:11.273222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff947c-915a-4aed-8a85-65bd11886c5b None admin] RBAC: Authorizing `identity:create_grant(project_id=811863e9c94d44a39647800f306505f0, user_id=dab0506091034403bf2f0fe5f7538cd6, role_id=c856bb63a85742ce8066968715c33568)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:11.276055 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff947c-915a-4aed-8a85-65bd11886c5b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:11.298321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2126c-aca5-4636-a5cf-07bfe2ebfa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:11.299478 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2126c-aca5-4636-a5cf-07bfe2ebfa06 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:11.299963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2126c-aca5-4636-a5cf-07bfe2ebfa06 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:11.300455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2126c-aca5-4636-a5cf-07bfe2ebfa06 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:11.301483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2126c-aca5-4636-a5cf-07bfe2ebfa06 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:11.304093 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2126c-aca5-4636-a5cf-07bfe2ebfa06 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1ed2126c-aca5-4636-a5cf-07bfe2ebfa06 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:11.324019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:11.325035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 08:27:11.325946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:11.327510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 470/940] 10.222.0.44 () {56 vars in 1132 bytes} [Sun Sep 11 08:27:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:11.328846 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2126c-aca5-4636-a5cf-07bfe2ebfa06 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:11.378995 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 471/941] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:27:11 2022] PUT /identity/v3/projects/811863e9c94d44a39647800f306505f0/users/dab0506091034403bf2f0fe5f7538cd6/roles/c856bb63a85742ce8066968715c33568 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:27:11.380913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff947c-915a-4aed-8a85-65bd11886c5b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:11.454180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:11.490783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2126c-aca5-4636-a5cf-07bfe2ebfa06 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:11.552605 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa265dfb-048d-4bc3-bcc7-9c24a7baccd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:11.553938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa265dfb-048d-4bc3-bcc7-9c24a7baccd1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:11.554447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa265dfb-048d-4bc3-bcc7-9c24a7baccd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:11.555006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa265dfb-048d-4bc3-bcc7-9c24a7baccd1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:11.556377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa265dfb-048d-4bc3-bcc7-9c24a7baccd1 None admin] RBAC: Adding query filter params (interface=public, service_id=bd0811a45d2f42f4a908057c95a330b6, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:11.557075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa265dfb-048d-4bc3-bcc7-9c24a7baccd1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:11.559634 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa265dfb-048d-4bc3-bcc7-9c24a7baccd1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:11.577400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 471/942] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 08:27:11 2022] GET /identity/v3/endpoints?service_id=bd0811a45d2f42f4a908057c95a330b6&interface=public®ion_id=RegionOne => generated 194 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:11.577400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa265dfb-048d-4bc3-bcc7-9c24a7baccd1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:11.605986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff947c-915a-4aed-8a85-65bd11886c5b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:11.696228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bbd34-ac36-476f-9b6e-a47d76a3464c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:11.696746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bbd34-ac36-476f-9b6e-a47d76a3464c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:11.697018 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bbd34-ac36-476f-9b6e-a47d76a3464c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:11.697284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bbd34-ac36-476f-9b6e-a47d76a3464c None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:11.697919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252bbd34-ac36-476f-9b6e-a47d76a3464c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:11.698849 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252bbd34-ac36-476f-9b6e-a47d76a3464c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:11.700275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252bbd34-ac36-476f-9b6e-a47d76a3464c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:11.707483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 472/943] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 08:27:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:11.713574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bbd34-ac36-476f-9b6e-a47d76a3464c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:11.764362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:11.765091 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa265dfb-048d-4bc3-bcc7-9c24a7baccd1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:11.826226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bbd34-ac36-476f-9b6e-a47d76a3464c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:11.827741 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067c3cb-8348-4d24-a305-9e06ff5cad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:11.828854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067c3cb-8348-4d24-a305-9e06ff5cad22 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:11.829380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067c3cb-8348-4d24-a305-9e06ff5cad22 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:11.829842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067c3cb-8348-4d24-a305-9e06ff5cad22 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:11.831046 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067c3cb-8348-4d24-a305-9e06ff5cad22 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:11.833686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067c3cb-8348-4d24-a305-9e06ff5cad22 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:11.838103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: WARNING keystone.server.flask.common [None req-f067c3cb-8348-4d24-a305-9e06ff5cad22 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 11 08:27:11.844908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:11.866255 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b7348-94d7-4129-a694-008a9c13fc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:11.866990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b7348-94d7-4129-a694-008a9c13fc2b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:11.867324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b7348-94d7-4129-a694-008a9c13fc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:11.867695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b7348-94d7-4129-a694-008a9c13fc2b None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:11.868371 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b7348-94d7-4129-a694-008a9c13fc2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:11.869760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b7348-94d7-4129-a694-008a9c13fc2b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:11.876754 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 473/944] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 08:27:11 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:27:11.877469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b7348-94d7-4129-a694-008a9c13fc2b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:11.903228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.sql.core [None req-f067c3cb-8348-4d24-a305-9e06ff5cad22 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 08:27:11.903228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 11 08:27:11.903228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [parameters: {'id': '051b874fbcb642db99740fe036eebb47', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 11 08:27:11.903228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81570) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 08:27:11.928658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f067c3cb-8348-4d24-a305-9e06ff5cad22 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 051b874fbcb642db99740fe036eebb47.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 051b874fbcb642db99740fe036eebb47. Sep 11 08:27:11.928658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:11.928658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 08:27:11.928658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 08:27:11.928658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 08:27:11.928658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 08:27:11.928658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 08:27:11.928658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 08:27:11.928658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 08:27:11.928658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application conn.query(q) Sep 11 08:27:11.928658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 08:27:11.928658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 08:27:11.928658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result.read() Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 11 08:27:11.929556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application next(self.gen) Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application yield resource Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application next(self.gen) Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise self.value Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application session.commit() Sep 11 08:27:11.930591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.session.flush() Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application compat.raise_( Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 08:27:11.931541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 08:27:11.934039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application conn.query(q) Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result.read() Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 08:27:11.936015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [parameters: {'id': '051b874fbcb642db99740fe036eebb47', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:11.938660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:11.941288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:11.943864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:11.943864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 11 08:27:11.943864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 11 08:27:11.943864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:11.943864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:11.943864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 11 08:27:11.943864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 11 08:27:11.943864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 051b874fbcb642db99740fe036eebb47. Sep 11 08:27:11.943864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:11.943864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 472/945] 10.222.0.44 () {60 vars in 1140 bytes} [Sun Sep 11 08:27:11 2022] POST /identity/v3/projects => generated 262 bytes in 368 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:11.943864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067c3cb-8348-4d24-a305-9e06ff5cad22 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:12.020869 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b7348-94d7-4129-a694-008a9c13fc2b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:12.036884 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:12.073426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2ff1a-108f-446b-9875-f78cb20a664c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:12.074053 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2ff1a-108f-446b-9875-f78cb20a664c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:12.075873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2ff1a-108f-446b-9875-f78cb20a664c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:12.076163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2ff1a-108f-446b-9875-f78cb20a664c None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:12.076836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2ff1a-108f-446b-9875-f78cb20a664c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:12.077135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2ff1a-108f-446b-9875-f78cb20a664c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:12.079148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2ff1a-108f-446b-9875-f78cb20a664c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:12.090475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 474/946] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 08:27:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:12.091486 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067c3cb-8348-4d24-a305-9e06ff5cad22 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:12.091910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2ff1a-108f-446b-9875-f78cb20a664c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:12.141554 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2906c67-a361-4032-9654-94dc40db7a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:12.144455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2906c67-a361-4032-9654-94dc40db7a03 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:12.145480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2906c67-a361-4032-9654-94dc40db7a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:12.145667 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2906c67-a361-4032-9654-94dc40db7a03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:12.155432 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2906c67-a361-4032-9654-94dc40db7a03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:12.157774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2906c67-a361-4032-9654-94dc40db7a03 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-a2906c67-a361-4032-9654-94dc40db7a03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:27:12.183441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:12.184430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:12.185497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:27:12.185497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:27:12.185497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:27:12.185497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:27:12.185497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:27:12.185497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:27:12.185497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:12.185497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:12.187709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 473/947] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:27:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:12.193165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2906c67-a361-4032-9654-94dc40db7a03 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:12.234493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2ff1a-108f-446b-9875-f78cb20a664c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:12.273183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:12.299135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352bd8b-e0b2-4eee-a875-3cf12c3435cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:12.300681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352bd8b-e0b2-4eee-a875-3cf12c3435cc None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:12.301445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352bd8b-e0b2-4eee-a875-3cf12c3435cc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:12.302181 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352bd8b-e0b2-4eee-a875-3cf12c3435cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:12.303674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352bd8b-e0b2-4eee-a875-3cf12c3435cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:12.306758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352bd8b-e0b2-4eee-a875-3cf12c3435cc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:12.339439 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2906c67-a361-4032-9654-94dc40db7a03 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:12.378930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53b6c2-7615-4173-b00a-515a37e28c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:12.379568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53b6c2-7615-4173-b00a-515a37e28c2b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:12.379830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53b6c2-7615-4173-b00a-515a37e28c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:12.380083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53b6c2-7615-4173-b00a-515a37e28c2b None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:12.380762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53b6c2-7615-4173-b00a-515a37e28c2b None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:12.381059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53b6c2-7615-4173-b00a-515a37e28c2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:12.382548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53b6c2-7615-4173-b00a-515a37e28c2b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:12.414192 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 474/948] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 08:27:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:12.442877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 475/949] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:27:12 2022] POST /identity/v3/endpoints => generated 347 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:27:13.944380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcf4f9-1c3d-49fd-be13-d916682968fb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:13.944924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcf4f9-1c3d-49fd-be13-d916682968fb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:13.944924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcf4f9-1c3d-49fd-be13-d916682968fb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:13.946568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 475/950] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:13.952511 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149bb95-c80f-4473-bcbb-7e6595948186 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:13.953603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149bb95-c80f-4473-bcbb-7e6595948186 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:13.953603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149bb95-c80f-4473-bcbb-7e6595948186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:13.973713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a149bb95-c80f-4473-bcbb-7e6595948186 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:13.975099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a149bb95-c80f-4473-bcbb-7e6595948186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:14.001988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:14.059064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 476/951] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:14.331173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcf4f9-1c3d-49fd-be13-d916682968fb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:14.451880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcf4f9-1c3d-49fd-be13-d916682968fb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:14.465152 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:14.492474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d8774-04f8-498e-b5d2-ed3ef1614f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:14.493600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d8774-04f8-498e-b5d2-ed3ef1614f73 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:14.494100 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d8774-04f8-498e-b5d2-ed3ef1614f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:14.494605 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d8774-04f8-498e-b5d2-ed3ef1614f73 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:14.508681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d8774-04f8-498e-b5d2-ed3ef1614f73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-638d8774-04f8-498e-b5d2-ed3ef1614f73 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:27:14.514206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:14.516804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:14.517709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:14.517709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:14.517709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:14.517709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:14.517709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:27:14.517709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:27:14.517709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:27:14.517709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:27:14.517709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:27:14.517709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:27:14.517709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 08:27:14.517709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:14.519299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 476/952] 10.222.0.44 () {56 vars in 1120 bytes} [Sun Sep 11 08:27:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:14.525184 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149bb95-c80f-4473-bcbb-7e6595948186 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:14.647903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149bb95-c80f-4473-bcbb-7e6595948186 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:14.686709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b05e0-5a37-46ff-b6e5-ece30c73ec59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:14.687368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b05e0-5a37-46ff-b6e5-ece30c73ec59 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:14.687680 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b05e0-5a37-46ff-b6e5-ece30c73ec59 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:14.687961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b05e0-5a37-46ff-b6e5-ece30c73ec59 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:14.688635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b05e0-5a37-46ff-b6e5-ece30c73ec59 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:14.688930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b05e0-5a37-46ff-b6e5-ece30c73ec59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:14.690348 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b05e0-5a37-46ff-b6e5-ece30c73ec59 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:14.700196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 477/953] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 08:27:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:14.705736 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d8774-04f8-498e-b5d2-ed3ef1614f73 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:14.761507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c4be6-06f2-46f5-ab61-f9351c9a1667 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:14.762010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c4be6-06f2-46f5-ab61-f9351c9a1667 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:14.762799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c4be6-06f2-46f5-ab61-f9351c9a1667 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:14.771790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 478/954] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:14 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:14.774068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b018c-8f75-4644-8aa8-eb52a3b51d0a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:14.774467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b018c-8f75-4644-8aa8-eb52a3b51d0a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:14.774931 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b018c-8f75-4644-8aa8-eb52a3b51d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:14.820644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5d2b018c-8f75-4644-8aa8-eb52a3b51d0a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:14.823479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5d2b018c-8f75-4644-8aa8-eb52a3b51d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:14.842640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d8774-04f8-498e-b5d2-ed3ef1614f73 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:14.854547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:14.914559 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45278b0f-fc91-4260-975c-b0afdec28a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:14.915671 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45278b0f-fc91-4260-975c-b0afdec28a9a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:14.916257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45278b0f-fc91-4260-975c-b0afdec28a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:14.916755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45278b0f-fc91-4260-975c-b0afdec28a9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:14.939117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45278b0f-fc91-4260-975c-b0afdec28a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:14.942061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45278b0f-fc91-4260-975c-b0afdec28a9a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:14.947708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 479/955] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-45278b0f-fc91-4260-975c-b0afdec28a9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:27:14.976643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:14.977983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:27:14.978914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:27:14.978914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:14.978914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:14.978914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 477/956] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:14.988494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b018c-8f75-4644-8aa8-eb52a3b51d0a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:15.182735 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b018c-8f75-4644-8aa8-eb52a3b51d0a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:15.219917 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079639c8-8d05-403c-9951-2526923fadc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:15.220842 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079639c8-8d05-403c-9951-2526923fadc6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:15.221294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079639c8-8d05-403c-9951-2526923fadc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:15.221761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079639c8-8d05-403c-9951-2526923fadc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:15.222628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079639c8-8d05-403c-9951-2526923fadc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:15.223120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079639c8-8d05-403c-9951-2526923fadc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:15.224722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079639c8-8d05-403c-9951-2526923fadc6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:15.228542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7b2a5-3f5c-4eff-8acb-45049ac21b47 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:15.228828 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7b2a5-3f5c-4eff-8acb-45049ac21b47 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:15.229100 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7b2a5-3f5c-4eff-8acb-45049ac21b47 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:15.230487 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 478/957] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:27:15.236051 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcf55d-c40d-4f3d-8bec-4225917e0b4f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:15.236467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcf55d-c40d-4f3d-8bec-4225917e0b4f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:15.236860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcf55d-c40d-4f3d-8bec-4225917e0b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:15.239627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 480/958] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:15.249591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079639c8-8d05-403c-9951-2526923fadc6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:15.256877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2fbcf55d-c40d-4f3d-8bec-4225917e0b4f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:15.258479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2fbcf55d-c40d-4f3d-8bec-4225917e0b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:15.263111 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:15.292615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:15.370794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 479/959] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:15.371691 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcf55d-c40d-4f3d-8bec-4225917e0b4f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:15.411801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079639c8-8d05-403c-9951-2526923fadc6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:15.450954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3014c8-3efe-4535-8973-c486339ba69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:15.451967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3014c8-3efe-4535-8973-c486339ba69a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:15.452508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3014c8-3efe-4535-8973-c486339ba69a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:15.453620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3014c8-3efe-4535-8973-c486339ba69a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:15.462617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3014c8-3efe-4535-8973-c486339ba69a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:15.464456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3014c8-3efe-4535-8973-c486339ba69a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:15.476791 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcf55d-c40d-4f3d-8bec-4225917e0b4f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0a3014c8-3efe-4535-8973-c486339ba69a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:27:15.477844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:15.478793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:15.479752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:15.479752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:15.479752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:27:15.479752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:27:15.479752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:27:15.479752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:27:15.479752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:27:15.479752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:27:15.479752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 08:27:15.479752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:15.482397 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 481/960] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 08:27:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:15.492700 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3014c8-3efe-4535-8973-c486339ba69a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:15.513051 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a72a18-bb75-41b5-9574-7b917a2eb601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:15.514311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a72a18-bb75-41b5-9574-7b917a2eb601 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:15.514311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a72a18-bb75-41b5-9574-7b917a2eb601 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:15.514311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a72a18-bb75-41b5-9574-7b917a2eb601 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:15.514861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a72a18-bb75-41b5-9574-7b917a2eb601 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:15.516929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a72a18-bb75-41b5-9574-7b917a2eb601 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-17a72a18-bb75-41b5-9574-7b917a2eb601 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:15.531096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:15.532246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 08:27:15.533120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:15.537390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 480/961] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:27:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:15.538511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a72a18-bb75-41b5-9574-7b917a2eb601 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:15.558364 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:15.595097 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:15.644778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3014c8-3efe-4535-8973-c486339ba69a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:15.655795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a72a18-bb75-41b5-9574-7b917a2eb601 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:15.702080 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3015172-134a-4d87-bcad-5fb26e50b2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:15.702917 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3015172-134a-4d87-bcad-5fb26e50b2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:15.703801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3015172-134a-4d87-bcad-5fb26e50b2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:15.704219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3015172-134a-4d87-bcad-5fb26e50b2b2 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:15.704402 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e63e5-5fc7-440f-8d86-8891ee64052e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:15.705153 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3015172-134a-4d87-bcad-5fb26e50b2b2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:15.705314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e63e5-5fc7-440f-8d86-8891ee64052e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:15.705631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3015172-134a-4d87-bcad-5fb26e50b2b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:15.705786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e63e5-5fc7-440f-8d86-8891ee64052e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:15.706275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e63e5-5fc7-440f-8d86-8891ee64052e None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:15.707473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e63e5-5fc7-440f-8d86-8891ee64052e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:15.707631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3015172-134a-4d87-bcad-5fb26e50b2b2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:15.708083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e63e5-5fc7-440f-8d86-8891ee64052e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:15.711499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e63e5-5fc7-440f-8d86-8891ee64052e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:15.720496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 481/962] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 08:27:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:15.722078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca828eb-baad-409c-ad24-54120f1c820d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:15.722580 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca828eb-baad-409c-ad24-54120f1c820d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:15.723024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca828eb-baad-409c-ad24-54120f1c820d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:15.725362 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 482/963] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:15.732370 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca828eb-baad-409c-ad24-54120f1c820d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:15.735949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 482/964] 10.222.0.44 () {56 vars in 1149 bytes} [Sun Sep 11 08:27:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:15.737616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118258e-24ed-4a73-b025-8957410efdd2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:15.738070 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118258e-24ed-4a73-b025-8957410efdd2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:15.740734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118258e-24ed-4a73-b025-8957410efdd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:15.769342 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7118258e-24ed-4a73-b025-8957410efdd2 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:15.771524 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7118258e-24ed-4a73-b025-8957410efdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:15.840433 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:15.881279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca828eb-baad-409c-ad24-54120f1c820d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:15.885447 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 483/965] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:15.886518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118258e-24ed-4a73-b025-8957410efdd2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:15.933823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de96659-af7b-497d-8606-aa29b1107219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:15.935482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de96659-af7b-497d-8606-aa29b1107219 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:15.937099 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de96659-af7b-497d-8606-aa29b1107219 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:15.938573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de96659-af7b-497d-8606-aa29b1107219 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:15.940039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de96659-af7b-497d-8606-aa29b1107219 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:15.940795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de96659-af7b-497d-8606-aa29b1107219 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:15.943465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de96659-af7b-497d-8606-aa29b1107219 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:15.953604 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 483/966] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 08:27:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:15.954957 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de96659-af7b-497d-8606-aa29b1107219 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:15.977381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:16.065074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118258e-24ed-4a73-b025-8957410efdd2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:16.137300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e94c56-d585-4fbb-9c2a-fba7d13e7d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:16.138315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e94c56-d585-4fbb-9c2a-fba7d13e7d1e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:16.138785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e94c56-d585-4fbb-9c2a-fba7d13e7d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:16.139245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e94c56-d585-4fbb-9c2a-fba7d13e7d1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:16.167021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de96659-af7b-497d-8606-aa29b1107219 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:16.246326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5072d-58e8-48b5-bcf6-4bcd43a33567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:16.247137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5072d-58e8-48b5-bcf6-4bcd43a33567 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:16.248018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5072d-58e8-48b5-bcf6-4bcd43a33567 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:16.248018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5072d-58e8-48b5-bcf6-4bcd43a33567 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:16.249005 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5072d-58e8-48b5-bcf6-4bcd43a33567 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:16.251671 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5072d-58e8-48b5-bcf6-4bcd43a33567 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:16.280002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e94c56-d585-4fbb-9c2a-fba7d13e7d1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:16.282491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e94c56-d585-4fbb-9c2a-fba7d13e7d1e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:16.319232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.sql.core [None req-8eb5072d-58e8-48b5-bcf6-4bcd43a33567 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 08:27:16.319232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 11 08:27:16.319232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [parameters: {'user_id': 'f3ece92278d54ed192a127af056702c1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 11 08:27:16.319232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81570) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8eb5072d-58e8-48b5-bcf6-4bcd43a33567 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application conn.query(q) Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result.read() Sep 11 08:27:16.337588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 08:27:16.338636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application next(self.gen) Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application yield resource Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application next(self.gen) Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise self.value Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application session.commit() Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 08:27:16.339905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.session.flush() Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application compat.raise_( Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 08:27:16.341257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 08:27:16.342195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application conn.query(q) Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result.read() Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 08:27:16.343160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f3ece92278d54ed192a127af056702c1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:16.344635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 11 08:27:16.345577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 11 08:27:16.346651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 11 08:27:16.346651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 11 08:27:16.346651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 08:27:16.346651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 08:27:16.346651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 11 08:27:16.346651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:16.346651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 484/967] 10.222.0.44 () {60 vars in 1131 bytes} [Sun Sep 11 08:27:15 2022] POST /identity/v3/users => generated 160 bytes in 387 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:16.354209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5072d-58e8-48b5-bcf6-4bcd43a33567 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:16.363297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:16.367843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:16.442379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 484/968] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:27:15 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:27:16.443843 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e94c56-d585-4fbb-9c2a-fba7d13e7d1e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:16.524534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5072d-58e8-48b5-bcf6-4bcd43a33567 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:16.586174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4f4db-09ca-48b9-b454-c00a843b594b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:16.587227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4f4db-09ca-48b9-b454-c00a843b594b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:16.587696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4f4db-09ca-48b9-b454-c00a843b594b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:16.588161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4f4db-09ca-48b9-b454-c00a843b594b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:16.602974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4f4db-09ca-48b9-b454-c00a843b594b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:16.606322 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4f4db-09ca-48b9-b454-c00a843b594b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:16.611048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e94c56-d585-4fbb-9c2a-fba7d13e7d1e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c2c4f4db-09ca-48b9-b454-c00a843b594b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:27:16.631406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:16.633278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:16.635217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:16.635217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:16.635217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:27:16.635217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:27:16.635217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:27:16.635217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:27:16.635217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:27:16.635217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:27:16.635217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 08:27:16.635217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:16.641617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 485/969] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:27:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:16.642947 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4f4db-09ca-48b9-b454-c00a843b594b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:16.655012 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079768d-c3ac-4f9b-8428-5082ce6bd1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:16.656726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079768d-c3ac-4f9b-8428-5082ce6bd1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:16.657142 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079768d-c3ac-4f9b-8428-5082ce6bd1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:16.657533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079768d-c3ac-4f9b-8428-5082ce6bd1a2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:16.658313 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079768d-c3ac-4f9b-8428-5082ce6bd1a2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:16.660219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079768d-c3ac-4f9b-8428-5082ce6bd1a2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d079768d-c3ac-4f9b-8428-5082ce6bd1a2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:16.670844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:16.672901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:16.675375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 485/970] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 08:27:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:16.676643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079768d-c3ac-4f9b-8428-5082ce6bd1a2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:16.704538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:16.772834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:16.833145 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4f4db-09ca-48b9-b454-c00a843b594b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:16.853808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079768d-c3ac-4f9b-8428-5082ce6bd1a2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:16.883950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d3828-add0-4dc4-b3b7-3184b8dcc9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:16.885042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d3828-add0-4dc4-b3b7-3184b8dcc9bf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:16.885542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d3828-add0-4dc4-b3b7-3184b8dcc9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:16.886024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d3828-add0-4dc4-b3b7-3184b8dcc9bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:16.905724 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14325f02-9b01-44d3-8983-b8b750fdf5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:16.906020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14325f02-9b01-44d3-8983-b8b750fdf5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:16.906299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14325f02-9b01-44d3-8983-b8b750fdf5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:16.906578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14325f02-9b01-44d3-8983-b8b750fdf5b9 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:16.907289 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14325f02-9b01-44d3-8983-b8b750fdf5b9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:16.907584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14325f02-9b01-44d3-8983-b8b750fdf5b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:16.909059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14325f02-9b01-44d3-8983-b8b750fdf5b9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:16.910257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d3828-add0-4dc4-b3b7-3184b8dcc9bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:16.913131 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d3828-add0-4dc4-b3b7-3184b8dcc9bf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ea7d3828-add0-4dc4-b3b7-3184b8dcc9bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:27:16.941826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:16.943178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:27:16.944670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:27:16.944670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:16.944670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:16.952543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 486/971] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:16.952543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d3828-add0-4dc4-b3b7-3184b8dcc9bf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:16.956471 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 486/972] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 08:27:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:16.958641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14325f02-9b01-44d3-8983-b8b750fdf5b9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:17.101028 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:17.136861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d3828-add0-4dc4-b3b7-3184b8dcc9bf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:17.147666 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14325f02-9b01-44d3-8983-b8b750fdf5b9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:17.158788 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:17.178324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf163810-82df-4b07-963e-0e1f912a193d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:17.179109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf163810-82df-4b07-963e-0e1f912a193d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:17.179705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf163810-82df-4b07-963e-0e1f912a193d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:17.179985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf163810-82df-4b07-963e-0e1f912a193d None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:17.180799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf163810-82df-4b07-963e-0e1f912a193d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:17.181110 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf163810-82df-4b07-963e-0e1f912a193d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:17.182696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf163810-82df-4b07-963e-0e1f912a193d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:17.194517 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 487/973] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 08:27:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:17.197653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf163810-82df-4b07-963e-0e1f912a193d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:17.218175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210dc50-6e03-4a31-877a-4a2928e8d4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:17.218652 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210dc50-6e03-4a31-877a-4a2928e8d4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:17.218832 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210dc50-6e03-4a31-877a-4a2928e8d4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:17.219069 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210dc50-6e03-4a31-877a-4a2928e8d4f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:17.219781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210dc50-6e03-4a31-877a-4a2928e8d4f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:17.220090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210dc50-6e03-4a31-877a-4a2928e8d4f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:17.221542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210dc50-6e03-4a31-877a-4a2928e8d4f0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:17.235512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 487/974] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:27:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:17.241366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210dc50-6e03-4a31-877a-4a2928e8d4f0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:17.348262 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf163810-82df-4b07-963e-0e1f912a193d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:17.392377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9198c3-48a9-415f-bb76-44327c7d6a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:17.393279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9198c3-48a9-415f-bb76-44327c7d6a16 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:17.393723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9198c3-48a9-415f-bb76-44327c7d6a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:17.394180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9198c3-48a9-415f-bb76-44327c7d6a16 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:17.394994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9198c3-48a9-415f-bb76-44327c7d6a16 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:17.395458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9198c3-48a9-415f-bb76-44327c7d6a16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:17.397736 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9198c3-48a9-415f-bb76-44327c7d6a16 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:17.403940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 488/975] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 08:27:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:17.411053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9198c3-48a9-415f-bb76-44327c7d6a16 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:17.417861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210dc50-6e03-4a31-877a-4a2928e8d4f0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:17.474330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0365e-22f4-4819-bb17-264a432c4099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:17.475148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0365e-22f4-4819-bb17-264a432c4099 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:17.475948 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0365e-22f4-4819-bb17-264a432c4099 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:17.476163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0365e-22f4-4819-bb17-264a432c4099 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:17.480008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:17.486259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:17.487518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0365e-22f4-4819-bb17-264a432c4099 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:17.490439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0365e-22f4-4819-bb17-264a432c4099 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6fc0365e-22f4-4819-bb17-264a432c4099 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:27:17.508493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:17.510338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:17.512040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:27:17.512040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:27:17.512040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:27:17.512040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:27:17.512040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:27:17.512040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:27:17.512040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:17.512040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:17.512040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 488/976] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:27:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:17.519057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0365e-22f4-4819-bb17-264a432c4099 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:17.576014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9198c3-48a9-415f-bb76-44327c7d6a16 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:17.640312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b2b3c-a288-403a-b261-ed42209a287e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:17.641347 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b2b3c-a288-403a-b261-ed42209a287e None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:17.641847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b2b3c-a288-403a-b261-ed42209a287e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:17.642450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b2b3c-a288-403a-b261-ed42209a287e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:17.643649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b2b3c-a288-403a-b261-ed42209a287e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:17.646614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b2b3c-a288-403a-b261-ed42209a287e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:17.675593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0365e-22f4-4819-bb17-264a432c4099 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:17.715151 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99620ad0-a8e1-4308-8c97-9a8930f76fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:17.715880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99620ad0-a8e1-4308-8c97-9a8930f76fae None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:17.716327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99620ad0-a8e1-4308-8c97-9a8930f76fae None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:17.716574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99620ad0-a8e1-4308-8c97-9a8930f76fae None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:17.717286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99620ad0-a8e1-4308-8c97-9a8930f76fae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:17.717587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99620ad0-a8e1-4308-8c97-9a8930f76fae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:17.719637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99620ad0-a8e1-4308-8c97-9a8930f76fae None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:17.741525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 489/977] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:27:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:17.743083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99620ad0-a8e1-4308-8c97-9a8930f76fae None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:17.763896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 489/978] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:27:17 2022] POST /identity/v3/endpoints => generated 349 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:27:17.819079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:17.892153 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99620ad0-a8e1-4308-8c97-9a8930f76fae None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:17.970672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902d1c8-458b-4d86-acb0-fde90d50df89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:17.971304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902d1c8-458b-4d86-acb0-fde90d50df89 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:17.971580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902d1c8-458b-4d86-acb0-fde90d50df89 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:17.971882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902d1c8-458b-4d86-acb0-fde90d50df89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:17.973070 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902d1c8-458b-4d86-acb0-fde90d50df89 None admin] RBAC: Adding query filter params (role.id=c856bb63a85742ce8066968715c33568, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=dab0506091034403bf2f0fe5f7538cd6) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:17.973526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902d1c8-458b-4d86-acb0-fde90d50df89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:17.975139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902d1c8-458b-4d86-acb0-fde90d50df89 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:17.984748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 490/979] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:27:17 2022] GET /identity/v3/role_assignments?user.id=dab0506091034403bf2f0fe5f7538cd6&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=c856bb63a85742ce8066968715c33568 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:18.560409 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36120b-7fd6-4fc1-8453-6bfc0f0f548a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:18.561063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36120b-7fd6-4fc1-8453-6bfc0f0f548a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:18.561063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36120b-7fd6-4fc1-8453-6bfc0f0f548a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:18.563059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 490/980] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:27:18.568658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68192fa-fd6d-43e4-a2e6-c802d1c14a41 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:18.569175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68192fa-fd6d-43e4-a2e6-c802d1c14a41 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:18.569175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68192fa-fd6d-43e4-a2e6-c802d1c14a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:18.589995 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e68192fa-fd6d-43e4-a2e6-c802d1c14a41 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:18.591547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e68192fa-fd6d-43e4-a2e6-c802d1c14a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:18.672077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 491/981] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:18.947531 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36120b-7fd6-4fc1-8453-6bfc0f0f548a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:19.113171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36120b-7fd6-4fc1-8453-6bfc0f0f548a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:19.115977 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:19.162032 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc46c88-4814-4dc7-a90a-908bdaa73071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:19.163020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc46c88-4814-4dc7-a90a-908bdaa73071 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:19.163560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc46c88-4814-4dc7-a90a-908bdaa73071 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:19.164097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc46c88-4814-4dc7-a90a-908bdaa73071 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:19.165101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc46c88-4814-4dc7-a90a-908bdaa73071 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:19.167268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc46c88-4814-4dc7-a90a-908bdaa73071 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:19.254182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 491/982] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:27:18 2022] POST /identity/v3/services => generated 242 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:27:20.060972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a7e6c-74cc-42e7-826f-1026f1225c91 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:20.061558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a7e6c-74cc-42e7-826f-1026f1225c91 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:20.061954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a7e6c-74cc-42e7-826f-1026f1225c91 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:20.064743 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 492/983] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:20.071185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161853f-fda9-4af4-a0f1-5116c68bb727 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:20.072118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161853f-fda9-4af4-a0f1-5116c68bb727 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:20.072118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161853f-fda9-4af4-a0f1-5116c68bb727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:20.092455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9161853f-fda9-4af4-a0f1-5116c68bb727 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:20.094643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9161853f-fda9-4af4-a0f1-5116c68bb727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:20.194085 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 492/984] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:27:20.448260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f25e86-b17a-447b-af63-5f315740db75 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:20.448446 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f25e86-b17a-447b-af63-5f315740db75 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:20.448722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f25e86-b17a-447b-af63-5f315740db75 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:20.450061 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 493/985] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:27:20.455867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372c476-bc70-4e27-9f26-48cd93920414 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:20.456151 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372c476-bc70-4e27-9f26-48cd93920414 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:20.456423 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372c476-bc70-4e27-9f26-48cd93920414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:20.477146 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2372c476-bc70-4e27-9f26-48cd93920414 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:20.478377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2372c476-bc70-4e27-9f26-48cd93920414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:20.540054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f25e86-b17a-447b-af63-5f315740db75 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:20.560530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 493/986] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:27:20.597020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:20.656849 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f25e86-b17a-447b-af63-5f315740db75 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:20.694719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794a8b9-9c67-4b83-b72d-0655cfcd07b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:20.695350 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794a8b9-9c67-4b83-b72d-0655cfcd07b9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:20.695620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794a8b9-9c67-4b83-b72d-0655cfcd07b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:20.695892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794a8b9-9c67-4b83-b72d-0655cfcd07b9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:20.696473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9794a8b9-9c67-4b83-b72d-0655cfcd07b9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:20.697853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9794a8b9-9c67-4b83-b72d-0655cfcd07b9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9794a8b9-9c67-4b83-b72d-0655cfcd07b9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:20.707418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:20.710541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:20.711441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 494/987] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 08:27:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:20.714576 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372c476-bc70-4e27-9f26-48cd93920414 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:20.737224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:20.842585 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372c476-bc70-4e27-9f26-48cd93920414 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:20.886867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3570b-002d-4157-a173-518b7548e028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:20.887490 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3570b-002d-4157-a173-518b7548e028 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:20.887921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3570b-002d-4157-a173-518b7548e028 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:20.888211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3570b-002d-4157-a173-518b7548e028 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:20.889105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3570b-002d-4157-a173-518b7548e028 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:20.889414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3570b-002d-4157-a173-518b7548e028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:20.891192 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3570b-002d-4157-a173-518b7548e028 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:20.902093 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 494/988] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 08:27:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:20.903125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794a8b9-9c67-4b83-b72d-0655cfcd07b9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:21.027902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794a8b9-9c67-4b83-b72d-0655cfcd07b9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:21.044232 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:21.070396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5ff3e-2d8a-49da-a9d7-6f5758aea0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:21.071275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5ff3e-2d8a-49da-a9d7-6f5758aea0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:21.071275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5ff3e-2d8a-49da-a9d7-6f5758aea0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:21.071643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5ff3e-2d8a-49da-a9d7-6f5758aea0c2 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:21.072171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5ff3e-2d8a-49da-a9d7-6f5758aea0c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:21.072468 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5ff3e-2d8a-49da-a9d7-6f5758aea0c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:21.073861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5ff3e-2d8a-49da-a9d7-6f5758aea0c2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:21.076374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3570b-002d-4157-a173-518b7548e028 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:21.089670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 495/989] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 08:27:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:21.091757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5ff3e-2d8a-49da-a9d7-6f5758aea0c2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:21.188499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:21.208392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3570b-002d-4157-a173-518b7548e028 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:21.231915 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5ff3e-2d8a-49da-a9d7-6f5758aea0c2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:21.248772 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fb770-6eaa-4bfd-8fc2-40f554d045cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:21.249417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fb770-6eaa-4bfd-8fc2-40f554d045cd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:21.249727 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fb770-6eaa-4bfd-8fc2-40f554d045cd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:21.249998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fb770-6eaa-4bfd-8fc2-40f554d045cd None admin] PATH_INFO: `/v3/services/image` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:21.250631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98fb770-6eaa-4bfd-8fc2-40f554d045cd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:21.252549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98fb770-6eaa-4bfd-8fc2-40f554d045cd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e98fb770-6eaa-4bfd-8fc2-40f554d045cd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:21.264795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:21.266662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:21.268448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 495/990] 10.222.0.44 () {56 vars in 1105 bytes} [Sun Sep 11 08:27:21 2022] GET /identity/v3/services/image => generated 86 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:21.272891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fb770-6eaa-4bfd-8fc2-40f554d045cd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:21.276245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd08fdf-a052-4da0-ba5c-a8d1519a8b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:21.277022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd08fdf-a052-4da0-ba5c-a8d1519a8b05 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:21.277301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd08fdf-a052-4da0-ba5c-a8d1519a8b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:21.277716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd08fdf-a052-4da0-ba5c-a8d1519a8b05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:21.278571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd08fdf-a052-4da0-ba5c-a8d1519a8b05 None admin] RBAC: Adding query filter params (interface=admin, service_id=bd0811a45d2f42f4a908057c95a330b6, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:21.278821 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd08fdf-a052-4da0-ba5c-a8d1519a8b05 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:21.280213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd08fdf-a052-4da0-ba5c-a8d1519a8b05 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:21.286667 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 496/991] 10.222.0.44 () {56 vars in 1249 bytes} [Sun Sep 11 08:27:21 2022] GET /identity/v3/endpoints?service_id=bd0811a45d2f42f4a908057c95a330b6&interface=admin®ion_id=RegionOne => generated 193 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:21.292459 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd08fdf-a052-4da0-ba5c-a8d1519a8b05 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:21.340839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:21.390903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fb770-6eaa-4bfd-8fc2-40f554d045cd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:21.410441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd08fdf-a052-4da0-ba5c-a8d1519a8b05 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:21.431276 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee38ed4-7927-4755-ab55-992cb18c9506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:21.431887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee38ed4-7927-4755-ab55-992cb18c9506 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:21.432166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee38ed4-7927-4755-ab55-992cb18c9506 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:21.432430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee38ed4-7927-4755-ab55-992cb18c9506 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:21.433063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee38ed4-7927-4755-ab55-992cb18c9506 None admin] RBAC: Adding query filter params (name=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:21.433359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee38ed4-7927-4755-ab55-992cb18c9506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:21.434767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee38ed4-7927-4755-ab55-992cb18c9506 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:21.440567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 496/992] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:21.447427 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee38ed4-7927-4755-ab55-992cb18c9506 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:21.460571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41b29a-5838-473c-aebf-5560236f56ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:21.461212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41b29a-5838-473c-aebf-5560236f56ce None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:21.461983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41b29a-5838-473c-aebf-5560236f56ce None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:21.461983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41b29a-5838-473c-aebf-5560236f56ce None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:21.462454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41b29a-5838-473c-aebf-5560236f56ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:21.464359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41b29a-5838-473c-aebf-5560236f56ce None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:21.465045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:21.478551 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 497/993] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 08:27:21 2022] GET /identity/v3/services => generated 1727 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:27:21.583481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee38ed4-7927-4755-ab55-992cb18c9506 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:21.594014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80191776-ecdc-4f11-93fc-1fe97068f863 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:21.594392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80191776-ecdc-4f11-93fc-1fe97068f863 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:21.594648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80191776-ecdc-4f11-93fc-1fe97068f863 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:21.596059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 498/994] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:27:21.602256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8d8a5-99c9-4e4c-a060-98964dab51fb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:21.602605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8d8a5-99c9-4e4c-a060-98964dab51fb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:21.602801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8d8a5-99c9-4e4c-a060-98964dab51fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:21.622568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-41e8d8a5-99c9-4e4c-a060-98964dab51fb None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:21.623803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-41e8d8a5-99c9-4e4c-a060-98964dab51fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:21.625518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548a283-58f9-48e7-93f8-88f9197895e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:21.626180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548a283-58f9-48e7-93f8-88f9197895e3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:21.626760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548a283-58f9-48e7-93f8-88f9197895e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:21.627041 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548a283-58f9-48e7-93f8-88f9197895e3 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:21.628708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548a283-58f9-48e7-93f8-88f9197895e3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:21.629017 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548a283-58f9-48e7-93f8-88f9197895e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:21.630526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548a283-58f9-48e7-93f8-88f9197895e3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:21.637731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 497/995] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:21.713519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 499/996] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:27:22.001689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548a283-58f9-48e7-93f8-88f9197895e3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:22.150338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:22.189902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548a283-58f9-48e7-93f8-88f9197895e3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:22.245851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6c1d0-f6bb-4a5c-aa00-79d60b3077f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:22.246930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6c1d0-f6bb-4a5c-aa00-79d60b3077f2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:22.247374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6c1d0-f6bb-4a5c-aa00-79d60b3077f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:22.247844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6c1d0-f6bb-4a5c-aa00-79d60b3077f2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:22.248927 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6c1d0-f6bb-4a5c-aa00-79d60b3077f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:22.251389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6c1d0-f6bb-4a5c-aa00-79d60b3077f2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4be6c1d0-f6bb-4a5c-aa00-79d60b3077f2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:22.268345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:22.269545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 08:27:22.270932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:22.274537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 498/997] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:27:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:22.279471 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8d8a5-99c9-4e4c-a060-98964dab51fb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:22.337385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:22.434276 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8d8a5-99c9-4e4c-a060-98964dab51fb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:22.481624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf54b9-3c84-4470-b0de-d4251e5d80ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:22.482269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf54b9-3c84-4470-b0de-d4251e5d80ee None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:22.482539 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf54b9-3c84-4470-b0de-d4251e5d80ee None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:22.482816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf54b9-3c84-4470-b0de-d4251e5d80ee None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:22.483451 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf54b9-3c84-4470-b0de-d4251e5d80ee None admin] RBAC: Adding query filter params (name=compute) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:22.483750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf54b9-3c84-4470-b0de-d4251e5d80ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:22.485185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf54b9-3c84-4470-b0de-d4251e5d80ee None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:22.491200 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 500/998] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 08:27:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:22.497883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6c1d0-f6bb-4a5c-aa00-79d60b3077f2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:22.656368 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6c1d0-f6bb-4a5c-aa00-79d60b3077f2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:22.698176 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01692e-f3e5-4533-90ed-d674eb6fbdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:22.698889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01692e-f3e5-4533-90ed-d674eb6fbdcd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:22.699355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01692e-f3e5-4533-90ed-d674eb6fbdcd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:22.699907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01692e-f3e5-4533-90ed-d674eb6fbdcd None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:22.700800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01692e-f3e5-4533-90ed-d674eb6fbdcd None admin] RBAC: Adding query filter params (type=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:22.701201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01692e-f3e5-4533-90ed-d674eb6fbdcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:22.703613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01692e-f3e5-4533-90ed-d674eb6fbdcd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:22.714209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 499/999] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 08:27:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:22.715713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf54b9-3c84-4470-b0de-d4251e5d80ee None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:22.837805 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:22.845914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf54b9-3c84-4470-b0de-d4251e5d80ee None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:22.909331 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ac4d1-316e-4ce8-a5ed-ff6181dd0f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:22.910664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ac4d1-316e-4ce8-a5ed-ff6181dd0f4b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:22.911256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ac4d1-316e-4ce8-a5ed-ff6181dd0f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:22.911879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ac4d1-316e-4ce8-a5ed-ff6181dd0f4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:22.913340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ac4d1-316e-4ce8-a5ed-ff6181dd0f4b None admin] RBAC: Adding query filter params (interface=public, service_id=15c26bfff08240aabf68c52c203f77a8, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:22.914071 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ac4d1-316e-4ce8-a5ed-ff6181dd0f4b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:22.916718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ac4d1-316e-4ce8-a5ed-ff6181dd0f4b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:22.926570 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 501/1000] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 08:27:22 2022] GET /identity/v3/endpoints?service_id=15c26bfff08240aabf68c52c203f77a8&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:22.934586 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01692e-f3e5-4533-90ed-d674eb6fbdcd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:22.939627 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:23.085035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01692e-f3e5-4533-90ed-d674eb6fbdcd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:23.123728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2148f6-c8f4-4b9c-b83e-d272a5a37ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:23.124536 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2148f6-c8f4-4b9c-b83e-d272a5a37ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:23.124814 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2148f6-c8f4-4b9c-b83e-d272a5a37ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:23.125104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2148f6-c8f4-4b9c-b83e-d272a5a37ca3 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:23.125755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2148f6-c8f4-4b9c-b83e-d272a5a37ca3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:23.127173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2148f6-c8f4-4b9c-b83e-d272a5a37ca3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:23.142264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 500/1001] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 08:27:22 2022] GET /identity/v3/services => generated 1727 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:27:23.767105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d07880-c662-4ba6-a50f-163fe611e681 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:23.767744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d07880-c662-4ba6-a50f-163fe611e681 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:23.767744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d07880-c662-4ba6-a50f-163fe611e681 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:23.769229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 502/1002] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:27:23.774711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851173f-3257-4947-b094-e5983d941795 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:23.775367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851173f-3257-4947-b094-e5983d941795 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:23.775367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851173f-3257-4947-b094-e5983d941795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:23.803274 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7851173f-3257-4947-b094-e5983d941795 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:23.805884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7851173f-3257-4947-b094-e5983d941795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:23.915931 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c529db-841a-407c-a847-0b90de9aebea None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:23.916276 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c529db-841a-407c-a847-0b90de9aebea None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:23.916473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c529db-841a-407c-a847-0b90de9aebea None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:23.917947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 503/1003] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:27:23.923714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65643cfe-34c9-4e59-a51a-40ebd5f00b98 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:23.924029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65643cfe-34c9-4e59-a51a-40ebd5f00b98 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:23.924257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65643cfe-34c9-4e59-a51a-40ebd5f00b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:23.925126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:23.938127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 501/1004] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:23 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:23.948134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-65643cfe-34c9-4e59-a51a-40ebd5f00b98 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:23.948134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-65643cfe-34c9-4e59-a51a-40ebd5f00b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:24.023772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 504/1005] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:23 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:27:24.228250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851173f-3257-4947-b094-e5983d941795 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:24.298372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65643cfe-34c9-4e59-a51a-40ebd5f00b98 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:24.380897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851173f-3257-4947-b094-e5983d941795 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:24.407413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:24.418903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f266c-13db-41a6-9d0e-97340bb2e5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:24.419738 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f266c-13db-41a6-9d0e-97340bb2e5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:24.421816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f266c-13db-41a6-9d0e-97340bb2e5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:24.421816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f266c-13db-41a6-9d0e-97340bb2e5a8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:24.421816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f266c-13db-41a6-9d0e-97340bb2e5a8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:24.422885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f266c-13db-41a6-9d0e-97340bb2e5a8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:24.424458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65643cfe-34c9-4e59-a51a-40ebd5f00b98 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c82f266c-13db-41a6-9d0e-97340bb2e5a8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:24.434847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:24.439044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:24.440183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 502/1006] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 08:27:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:24.443072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f266c-13db-41a6-9d0e-97340bb2e5a8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:24.484941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d68161-9960-4360-a539-465866ee656a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:24.486045 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d68161-9960-4360-a539-465866ee656a None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:24.486557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d68161-9960-4360-a539-465866ee656a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:24.487013 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d68161-9960-4360-a539-465866ee656a None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:24.488184 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d68161-9960-4360-a539-465866ee656a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:24.490734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d68161-9960-4360-a539-465866ee656a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:24.522985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:24.565778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f266c-13db-41a6-9d0e-97340bb2e5a8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:24.592004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 505/1007] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:27:24 2022] POST /identity/v3/services => generated 242 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:27:24.616243 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af568260-9431-40b3-83f5-06642e30888f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:24.617201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af568260-9431-40b3-83f5-06642e30888f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:24.617201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af568260-9431-40b3-83f5-06642e30888f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:24.617444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af568260-9431-40b3-83f5-06642e30888f None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:24.618255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af568260-9431-40b3-83f5-06642e30888f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:24.622434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af568260-9431-40b3-83f5-06642e30888f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:24.622434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af568260-9431-40b3-83f5-06642e30888f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:24.629566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 503/1008] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 08:27:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:24.638529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d68161-9960-4360-a539-465866ee656a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:24.823049 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d68161-9960-4360-a539-465866ee656a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:24.864288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8665506-34af-4da8-88ed-24ddd4ab0b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:24.864915 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8665506-34af-4da8-88ed-24ddd4ab0b7d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:24.865189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8665506-34af-4da8-88ed-24ddd4ab0b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:24.865459 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8665506-34af-4da8-88ed-24ddd4ab0b7d None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:24.866100 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8665506-34af-4da8-88ed-24ddd4ab0b7d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:24.866438 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8665506-34af-4da8-88ed-24ddd4ab0b7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:24.867834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8665506-34af-4da8-88ed-24ddd4ab0b7d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:24.878886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 506/1009] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 08:27:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:24.888610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af568260-9431-40b3-83f5-06642e30888f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:25.066703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af568260-9431-40b3-83f5-06642e30888f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:25.096249 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:25.117396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324dc23-89bb-46ba-a0ff-55e27f28ef5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:25.118481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324dc23-89bb-46ba-a0ff-55e27f28ef5a None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:25.118896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324dc23-89bb-46ba-a0ff-55e27f28ef5a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:25.123020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324dc23-89bb-46ba-a0ff-55e27f28ef5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:25.123020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324dc23-89bb-46ba-a0ff-55e27f28ef5a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:25.123020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324dc23-89bb-46ba-a0ff-55e27f28ef5a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:25.218668 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 504/1010] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:27:24 2022] POST /identity/v3/endpoints => generated 325 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:27:25.649227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f00e1c-c933-4a21-a32e-d883075c6a1e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:25.649577 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f00e1c-c933-4a21-a32e-d883075c6a1e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:25.649797 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f00e1c-c933-4a21-a32e-d883075c6a1e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:25.651325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 507/1011] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:25.657332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30771422-fae5-4e15-938e-c6b0b996f8a8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:25.657698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30771422-fae5-4e15-938e-c6b0b996f8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:25.657858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30771422-fae5-4e15-938e-c6b0b996f8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:25.677903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-30771422-fae5-4e15-938e-c6b0b996f8a8 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:25.679137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-30771422-fae5-4e15-938e-c6b0b996f8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:25.777303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 505/1012] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:25 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:26.046596 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f00e1c-c933-4a21-a32e-d883075c6a1e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:26.065718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:26.164599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f00e1c-c933-4a21-a32e-d883075c6a1e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:26.202014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056df20-9f35-4035-8f4a-7f7de6ba1822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:26.203277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056df20-9f35-4035-8f4a-7f7de6ba1822 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:26.203559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056df20-9f35-4035-8f4a-7f7de6ba1822 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:26.204542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056df20-9f35-4035-8f4a-7f7de6ba1822 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:26.205150 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056df20-9f35-4035-8f4a-7f7de6ba1822 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:26.206582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056df20-9f35-4035-8f4a-7f7de6ba1822 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5056df20-9f35-4035-8f4a-7f7de6ba1822 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:26.216576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:26.217718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 08:27:26.218717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:26.222614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 508/1013] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:27:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:26.224118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30771422-fae5-4e15-938e-c6b0b996f8a8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:26.342782 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30771422-fae5-4e15-938e-c6b0b996f8a8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:26.351624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:26.382848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9239cf-d7c6-4159-b828-e6813c93bb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:26.383462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9239cf-d7c6-4159-b828-e6813c93bb09 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:26.383742 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9239cf-d7c6-4159-b828-e6813c93bb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:26.384045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9239cf-d7c6-4159-b828-e6813c93bb09 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:26.384776 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9239cf-d7c6-4159-b828-e6813c93bb09 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:26.385286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9239cf-d7c6-4159-b828-e6813c93bb09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:26.387934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9239cf-d7c6-4159-b828-e6813c93bb09 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:26.395242 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 506/1014] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 08:27:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:26.401736 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056df20-9f35-4035-8f4a-7f7de6ba1822 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:26.509545 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:26.538443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056df20-9f35-4035-8f4a-7f7de6ba1822 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:26.581155 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c92860-7c3b-4c6b-91d7-285d80c862a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:26.581701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c92860-7c3b-4c6b-91d7-285d80c862a0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:26.581985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c92860-7c3b-4c6b-91d7-285d80c862a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:26.582281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c92860-7c3b-4c6b-91d7-285d80c862a0 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:26.582910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c92860-7c3b-4c6b-91d7-285d80c862a0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:26.583247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c92860-7c3b-4c6b-91d7-285d80c862a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:26.584640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c92860-7c3b-4c6b-91d7-285d80c862a0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:26.593884 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 509/1015] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 08:27:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:26.596979 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9239cf-d7c6-4159-b828-e6813c93bb09 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:26.720294 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9239cf-d7c6-4159-b828-e6813c93bb09 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:26.759376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbf38c-af2c-4fd2-8ce9-fb9296392892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:26.760005 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbf38c-af2c-4fd2-8ce9-fb9296392892 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:26.760282 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbf38c-af2c-4fd2-8ce9-fb9296392892 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:26.760547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbf38c-af2c-4fd2-8ce9-fb9296392892 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:26.761216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbf38c-af2c-4fd2-8ce9-fb9296392892 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:26.762622 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbf38c-af2c-4fd2-8ce9-fb9296392892 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:26.835567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 507/1016] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:27:26 2022] POST /identity/v3/endpoints => generated 334 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:27:26.940245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34762d-ad25-4f7c-9f99-2c78a7d8404f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:26.940595 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34762d-ad25-4f7c-9f99-2c78a7d8404f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:26.940957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34762d-ad25-4f7c-9f99-2c78a7d8404f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:26.942854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 510/1017] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:27:26.949881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31870b7-4652-4dfc-8197-6a0fbee08ca2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:26.950359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31870b7-4652-4dfc-8197-6a0fbee08ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:26.950530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31870b7-4652-4dfc-8197-6a0fbee08ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:26.979358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c31870b7-4652-4dfc-8197-6a0fbee08ca2 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:26.980769 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c31870b7-4652-4dfc-8197-6a0fbee08ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:27.005720 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:27.075357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 508/1018] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:27.355358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34762d-ad25-4f7c-9f99-2c78a7d8404f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:27.474408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34762d-ad25-4f7c-9f99-2c78a7d8404f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:27.519922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d169d-6959-40fb-ae9c-badb72f7bfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:27.520760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d169d-6959-40fb-ae9c-badb72f7bfaf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:27.521167 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d169d-6959-40fb-ae9c-badb72f7bfaf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:27.521771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d169d-6959-40fb-ae9c-badb72f7bfaf None admin] PATH_INFO: `/v3/services/image` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:27.522566 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d169d-6959-40fb-ae9c-badb72f7bfaf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:27.524120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d169d-6959-40fb-ae9c-badb72f7bfaf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-991d169d-6959-40fb-ae9c-badb72f7bfaf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:27.534834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:27.538402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:27.540149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 511/1019] 10.222.0.44 () {56 vars in 1105 bytes} [Sun Sep 11 08:27:27 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:27.545212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31870b7-4652-4dfc-8197-6a0fbee08ca2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:27.616076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968224b-f39b-488d-8c26-88bf4ccba327 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:27.616296 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968224b-f39b-488d-8c26-88bf4ccba327 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:27.616532 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968224b-f39b-488d-8c26-88bf4ccba327 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:27.617907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 512/1020] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:27:27.623247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de758d7-76bb-4d83-a3d1-c939c6d01995 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:27.623533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de758d7-76bb-4d83-a3d1-c939c6d01995 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:27.623821 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de758d7-76bb-4d83-a3d1-c939c6d01995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:27.637399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:27.644949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0de758d7-76bb-4d83-a3d1-c939c6d01995 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:27.646267 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0de758d7-76bb-4d83-a3d1-c939c6d01995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:27.709953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31870b7-4652-4dfc-8197-6a0fbee08ca2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:27.734245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 513/1021] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:27.752986 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfa114-a289-4da0-ab11-dc1caffd502d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:27.754351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfa114-a289-4da0-ab11-dc1caffd502d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:27.755066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfa114-a289-4da0-ab11-dc1caffd502d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:27.755783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfa114-a289-4da0-ab11-dc1caffd502d None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:27.757166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfa114-a289-4da0-ab11-dc1caffd502d None admin] RBAC: Adding query filter params (name=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:27.757953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfa114-a289-4da0-ab11-dc1caffd502d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:27.760873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfa114-a289-4da0-ab11-dc1caffd502d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:27.777790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 509/1022] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:27.779841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de758d7-76bb-4d83-a3d1-c939c6d01995 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:27.912765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de758d7-76bb-4d83-a3d1-c939c6d01995 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:27.952168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621003d-72b9-4d90-8f77-fde6c719d99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:27.952983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621003d-72b9-4d90-8f77-fde6c719d99a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:27.953400 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621003d-72b9-4d90-8f77-fde6c719d99a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:27.953816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621003d-72b9-4d90-8f77-fde6c719d99a None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:27.954634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621003d-72b9-4d90-8f77-fde6c719d99a None admin] RBAC: Adding query filter params (type=image) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:27.955060 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621003d-72b9-4d90-8f77-fde6c719d99a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:27.956567 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621003d-72b9-4d90-8f77-fde6c719d99a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:27.959636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:27.963627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 514/1023] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:27.971380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfa114-a289-4da0-ab11-dc1caffd502d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:28.006956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:28.035845 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621003d-72b9-4d90-8f77-fde6c719d99a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:28.126724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfa114-a289-4da0-ab11-dc1caffd502d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:28.167589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfd9c6-ff9e-4980-910d-bf143c10cb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:28.168234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfd9c6-ff9e-4980-910d-bf143c10cb21 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:28.168537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfd9c6-ff9e-4980-910d-bf143c10cb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:28.168814 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfd9c6-ff9e-4980-910d-bf143c10cb21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:28.169487 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfd9c6-ff9e-4980-910d-bf143c10cb21 None admin] RBAC: Adding query filter params (interface=public, service_id=853bd823e37245e6be96db4f6b11ee33, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:28.169790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfd9c6-ff9e-4980-910d-bf143c10cb21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:28.171922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfd9c6-ff9e-4980-910d-bf143c10cb21 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:28.182223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 510/1024] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 08:27:27 2022] GET /identity/v3/endpoints?service_id=853bd823e37245e6be96db4f6b11ee33&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:28.183921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfd9c6-ff9e-4980-910d-bf143c10cb21 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:28.198699 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621003d-72b9-4d90-8f77-fde6c719d99a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:28.257779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed283f9-ccbb-44b7-910a-6455d48bae23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:28.258915 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed283f9-ccbb-44b7-910a-6455d48bae23 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:28.259382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed283f9-ccbb-44b7-910a-6455d48bae23 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:28.259835 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed283f9-ccbb-44b7-910a-6455d48bae23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:28.261986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed283f9-ccbb-44b7-910a-6455d48bae23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:28.264851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed283f9-ccbb-44b7-910a-6455d48bae23 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:28.268709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 515/1025] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:28.274556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed283f9-ccbb-44b7-910a-6455d48bae23 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:28.282171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:28.316997 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfd9c6-ff9e-4980-910d-bf143c10cb21 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:28.347697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:28.382806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed283f9-ccbb-44b7-910a-6455d48bae23 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:28.384861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e944c-9cff-4497-9f15-3682b9c2ce0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:28.386122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e944c-9cff-4497-9f15-3682b9c2ce0c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:28.386870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e944c-9cff-4497-9f15-3682b9c2ce0c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:28.387571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e944c-9cff-4497-9f15-3682b9c2ce0c None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:28.388894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e944c-9cff-4497-9f15-3682b9c2ce0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:28.391714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e944c-9cff-4497-9f15-3682b9c2ce0c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:28.406801 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 511/1026] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 08:27:28 2022] GET /identity/v3/services => generated 1957 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:27:28.425845 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf5b55-6b72-4272-a58d-b099e1a7c39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:28.426443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf5b55-6b72-4272-a58d-b099e1a7c39b None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:28.426716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf5b55-6b72-4272-a58d-b099e1a7c39b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:28.426982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf5b55-6b72-4272-a58d-b099e1a7c39b None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:28.429137 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf5b55-6b72-4272-a58d-b099e1a7c39b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:28.429338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf5b55-6b72-4272-a58d-b099e1a7c39b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:28.532865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 516/1027] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:27:28 2022] POST /identity/v3/projects => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:27:29.319521 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5552580-65e3-4dee-b695-320115ded70f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:29.320209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5552580-65e3-4dee-b695-320115ded70f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:29.320209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5552580-65e3-4dee-b695-320115ded70f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:29.321691 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 512/1028] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:29.328063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b7115-e2fc-4243-a162-be26efa82307 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:29.328386 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b7115-e2fc-4243-a162-be26efa82307 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:29.328614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b7115-e2fc-4243-a162-be26efa82307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:29.350139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8d5b7115-e2fc-4243-a162-be26efa82307 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:29.351401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8d5b7115-e2fc-4243-a162-be26efa82307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:29.437499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 517/1029] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:29.706292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5552580-65e3-4dee-b695-320115ded70f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:29.842504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5552580-65e3-4dee-b695-320115ded70f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:29.909445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0801878-8f18-4285-8019-4a0a4a5e02b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:29.910090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801878-8f18-4285-8019-4a0a4a5e02b2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:29.910506 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801878-8f18-4285-8019-4a0a4a5e02b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:29.910893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801878-8f18-4285-8019-4a0a4a5e02b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:29.928102 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0801878-8f18-4285-8019-4a0a4a5e02b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f0801878-8f18-4285-8019-4a0a4a5e02b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:27:29.938255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:29.939290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:29.943301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:29.943301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:29.943301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:29.943301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:29.943301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:27:29.943301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:27:29.943301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:27:29.943301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:27:29.943301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:27:29.943301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:27:29.943301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 08:27:29.943301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:29.943301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 513/1030] 10.222.0.44 () {56 vars in 1120 bytes} [Sun Sep 11 08:27:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:29.949113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b7115-e2fc-4243-a162-be26efa82307 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:30.018312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:30.098947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b7115-e2fc-4243-a162-be26efa82307 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:30.160841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c403faf-ffd5-47c1-96fe-81856cf39845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:30.161458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c403faf-ffd5-47c1-96fe-81856cf39845 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:30.161732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c403faf-ffd5-47c1-96fe-81856cf39845 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:30.161993 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c403faf-ffd5-47c1-96fe-81856cf39845 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:30.162710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c403faf-ffd5-47c1-96fe-81856cf39845 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:30.163006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c403faf-ffd5-47c1-96fe-81856cf39845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:30.164401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c403faf-ffd5-47c1-96fe-81856cf39845 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:30.174599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 518/1031] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 08:27:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:30.180674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0801878-8f18-4285-8019-4a0a4a5e02b2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:30.185470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:30.293374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0801878-8f18-4285-8019-4a0a4a5e02b2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:30.332663 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e2e4f-41ac-4883-9848-15d7a4db1b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:30.333646 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e2e4f-41ac-4883-9848-15d7a4db1b46 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:30.333926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e2e4f-41ac-4883-9848-15d7a4db1b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:30.334237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e2e4f-41ac-4883-9848-15d7a4db1b46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:30.345877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e2e4f-41ac-4883-9848-15d7a4db1b46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:30.347364 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e2e4f-41ac-4883-9848-15d7a4db1b46 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ee8e2e4f-41ac-4883-9848-15d7a4db1b46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:27:30.363829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:30.365087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:27:30.366844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:27:30.366844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:30.366844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:30.366844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 514/1032] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:30.371449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c403faf-ffd5-47c1-96fe-81856cf39845 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:30.516510 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c403faf-ffd5-47c1-96fe-81856cf39845 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:30.537030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:30.571385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa33fe0-d352-444e-acb6-bc4fed567d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:30.572358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa33fe0-d352-444e-acb6-bc4fed567d5e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:30.572850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa33fe0-d352-444e-acb6-bc4fed567d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:30.573326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa33fe0-d352-444e-acb6-bc4fed567d5e None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:30.574609 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa33fe0-d352-444e-acb6-bc4fed567d5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:30.575155 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa33fe0-d352-444e-acb6-bc4fed567d5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:30.578423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa33fe0-d352-444e-acb6-bc4fed567d5e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:30.608005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 519/1033] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:30.610129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e2e4f-41ac-4883-9848-15d7a4db1b46 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:30.700728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:30.701457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007229-1e4f-4f29-a3fb-139ef6969541 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:30.701744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007229-1e4f-4f29-a3fb-139ef6969541 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:30.702020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007229-1e4f-4f29-a3fb-139ef6969541 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:30.704322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 520/1034] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:27:30.709852 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f4f10-1c15-4fa0-891f-2b9dedd01bca None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:30.710191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f4f10-1c15-4fa0-891f-2b9dedd01bca None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:30.710430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f4f10-1c15-4fa0-891f-2b9dedd01bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:30.729199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b19f4f10-1c15-4fa0-891f-2b9dedd01bca None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:30.730458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b19f4f10-1c15-4fa0-891f-2b9dedd01bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:30.755632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e2e4f-41ac-4883-9848-15d7a4db1b46 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:30.795531 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab26ca5-46aa-4736-b0c0-9d1eb1ee68b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:30.796707 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab26ca5-46aa-4736-b0c0-9d1eb1ee68b4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:30.796988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab26ca5-46aa-4736-b0c0-9d1eb1ee68b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:30.797261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab26ca5-46aa-4736-b0c0-9d1eb1ee68b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:30.807405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab26ca5-46aa-4736-b0c0-9d1eb1ee68b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:30.808861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab26ca5-46aa-4736-b0c0-9d1eb1ee68b4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0ab26ca5-46aa-4736-b0c0-9d1eb1ee68b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:27:30.825275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:30.826747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:30.828081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:30.828081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:30.828081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:27:30.828081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:27:30.828081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:27:30.828081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:27:30.828081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:27:30.828081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:27:30.828081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:27:30.828081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:30.831286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 515/1035] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 08:27:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:30.832445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab26ca5-46aa-4736-b0c0-9d1eb1ee68b4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:30.839035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 521/1036] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:27:30.979560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab26ca5-46aa-4736-b0c0-9d1eb1ee68b4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:31.002783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:31.028728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8ce8a-f66b-48ce-bd91-8e7d3602c64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:31.032639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8ce8a-f66b-48ce-bd91-8e7d3602c64e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:31.032639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8ce8a-f66b-48ce-bd91-8e7d3602c64e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:31.032639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8ce8a-f66b-48ce-bd91-8e7d3602c64e None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:31.032639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8ce8a-f66b-48ce-bd91-8e7d3602c64e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:31.032639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8ce8a-f66b-48ce-bd91-8e7d3602c64e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:31.033352 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8ce8a-f66b-48ce-bd91-8e7d3602c64e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:31.043038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f583d6-5c9c-40b2-9316-8f40385e0a13 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:31.043300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f583d6-5c9c-40b2-9316-8f40385e0a13 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:31.043599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f583d6-5c9c-40b2-9316-8f40385e0a13 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:31.044944 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 522/1037] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:27:31.051646 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feba931-1199-4fd2-9087-550144fe6a34 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:31.051943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feba931-1199-4fd2-9087-550144fe6a34 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:31.052293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feba931-1199-4fd2-9087-550144fe6a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:31.059100 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 516/1038] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 08:27:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:31.065434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8ce8a-f66b-48ce-bd91-8e7d3602c64e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:31.075058 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8feba931-1199-4fd2-9087-550144fe6a34 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:31.076353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8feba931-1199-4fd2-9087-550144fe6a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:31.180376 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 523/1039] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:27:31.181192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feba931-1199-4fd2-9087-550144fe6a34 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:31.218547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8ce8a-f66b-48ce-bd91-8e7d3602c64e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:31.230040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:31.259396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afa89a-7118-45b5-a554-f57d7dbad6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:31.260078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa89a-7118-45b5-a554-f57d7dbad6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:31.260384 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa89a-7118-45b5-a554-f57d7dbad6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:31.260625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa89a-7118-45b5-a554-f57d7dbad6d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:31.325817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:31.335243 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feba931-1199-4fd2-9087-550144fe6a34 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:31.364033 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afa89a-7118-45b5-a554-f57d7dbad6d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:31.366667 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afa89a-7118-45b5-a554-f57d7dbad6d7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:31.385141 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1a77b-a9b3-4b24-b81a-d46e0bce503b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:31.386471 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1a77b-a9b3-4b24-b81a-d46e0bce503b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:31.386600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1a77b-a9b3-4b24-b81a-d46e0bce503b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:31.386765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1a77b-a9b3-4b24-b81a-d46e0bce503b None admin] PATH_INFO: `/v3/services/image` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:31.387347 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1a77b-a9b3-4b24-b81a-d46e0bce503b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:31.389048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1a77b-a9b3-4b24-b81a-d46e0bce503b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9aa1a77b-a9b3-4b24-b81a-d46e0bce503b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:31.400432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:31.401556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 08:27:31.402873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:31.406610 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 524/1040] 10.222.0.44 () {56 vars in 1105 bytes} [Sun Sep 11 08:27:31 2022] GET /identity/v3/services/image => generated 86 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:31.408433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1a77b-a9b3-4b24-b81a-d46e0bce503b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:31.516186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:31.522796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1a77b-a9b3-4b24-b81a-d46e0bce503b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:31.536234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 517/1041] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:27:31 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 08:27:31.536234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afa89a-7118-45b5-a554-f57d7dbad6d7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:31.566649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a2e70-a261-49ca-a722-fc688061e2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:31.567259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a2e70-a261-49ca-a722-fc688061e2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:31.567538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a2e70-a261-49ca-a722-fc688061e2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:31.567809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a2e70-a261-49ca-a722-fc688061e2d7 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:31.568463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a2e70-a261-49ca-a722-fc688061e2d7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:31.568782 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a2e70-a261-49ca-a722-fc688061e2d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:31.570181 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a2e70-a261-49ca-a722-fc688061e2d7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:31.577241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 525/1042] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:31.579622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a2e70-a261-49ca-a722-fc688061e2d7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:31.641229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:31.670916 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afa89a-7118-45b5-a554-f57d7dbad6d7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:31.712596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294864d-8cbc-46d1-a0d9-40d4970e7708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:31.713253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294864d-8cbc-46d1-a0d9-40d4970e7708 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:31.713531 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294864d-8cbc-46d1-a0d9-40d4970e7708 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:31.713800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294864d-8cbc-46d1-a0d9-40d4970e7708 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:31.715107 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294864d-8cbc-46d1-a0d9-40d4970e7708 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:31.716896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294864d-8cbc-46d1-a0d9-40d4970e7708 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:31.719407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 518/1043] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:31.720044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294864d-8cbc-46d1-a0d9-40d4970e7708 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:31.720197 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a2e70-a261-49ca-a722-fc688061e2d7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:31.756799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aef1e5-3f89-4f45-b3ad-1bfed5f680fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:31.757574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aef1e5-3f89-4f45-b3ad-1bfed5f680fc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:31.757940 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aef1e5-3f89-4f45-b3ad-1bfed5f680fc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:31.758348 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aef1e5-3f89-4f45-b3ad-1bfed5f680fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:31.770234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aef1e5-3f89-4f45-b3ad-1bfed5f680fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:31.771698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aef1e5-3f89-4f45-b3ad-1bfed5f680fc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-17aef1e5-3f89-4f45-b3ad-1bfed5f680fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:27:31.787164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:31.792958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:27:31.794125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:27:31.794125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:31.794125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:31.794125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 526/1044] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:31.794125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aef1e5-3f89-4f45-b3ad-1bfed5f680fc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:31.805159 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:31.839392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294864d-8cbc-46d1-a0d9-40d4970e7708 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:31.879743 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bac6ee-5e4b-43fb-907b-d26966ad867d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:31.880584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bac6ee-5e4b-43fb-907b-d26966ad867d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:31.880920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bac6ee-5e4b-43fb-907b-d26966ad867d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:31.881307 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bac6ee-5e4b-43fb-907b-d26966ad867d None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:31.882053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bac6ee-5e4b-43fb-907b-d26966ad867d None admin] RBAC: Adding query filter params (type=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:31.882429 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bac6ee-5e4b-43fb-907b-d26966ad867d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:31.884268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bac6ee-5e4b-43fb-907b-d26966ad867d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:31.891130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 519/1045] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:31.892384 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bac6ee-5e4b-43fb-907b-d26966ad867d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:31.902103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:31.909631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aef1e5-3f89-4f45-b3ad-1bfed5f680fc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:31.951211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f8de3-9272-41cf-8d3b-192bda48821e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:31.951813 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f8de3-9272-41cf-8d3b-192bda48821e None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:31.952086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f8de3-9272-41cf-8d3b-192bda48821e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:31.952409 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f8de3-9272-41cf-8d3b-192bda48821e None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:31.953562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f8de3-9272-41cf-8d3b-192bda48821e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:31.955035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f8de3-9272-41cf-8d3b-192bda48821e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:32.002925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bac6ee-5e4b-43fb-907b-d26966ad867d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:32.036831 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf89686-dc93-42fe-958c-12e53e985a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:32.037439 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf89686-dc93-42fe-958c-12e53e985a0c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:32.037717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf89686-dc93-42fe-958c-12e53e985a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:32.038005 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf89686-dc93-42fe-958c-12e53e985a0c None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:32.038661 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf89686-dc93-42fe-958c-12e53e985a0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:32.038965 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf89686-dc93-42fe-958c-12e53e985a0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:32.040320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf89686-dc93-42fe-958c-12e53e985a0c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:32.052346 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 520/1046] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:32.053142 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf89686-dc93-42fe-958c-12e53e985a0c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:32.062861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 527/1047] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:27:31 2022] POST /identity/v3/users => generated 285 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:27:32.063694 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f8de3-9272-41cf-8d3b-192bda48821e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:32.161173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:32.176441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:32.224197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf89686-dc93-42fe-958c-12e53e985a0c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:32.239216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f8de3-9272-41cf-8d3b-192bda48821e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:32.277065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf18a9-e12a-4f41-8c9b-1899d7ccd599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:32.277703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf18a9-e12a-4f41-8c9b-1899d7ccd599 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:32.277981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf18a9-e12a-4f41-8c9b-1899d7ccd599 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:32.278359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf18a9-e12a-4f41-8c9b-1899d7ccd599 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:32.279049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf18a9-e12a-4f41-8c9b-1899d7ccd599 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:32.280497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf18a9-e12a-4f41-8c9b-1899d7ccd599 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:32.296782 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdf2c4-22b0-4a7f-b8d0-eae805debefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:32.302098 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdf2c4-22b0-4a7f-b8d0-eae805debefd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:32.302098 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdf2c4-22b0-4a7f-b8d0-eae805debefd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:32.302098 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdf2c4-22b0-4a7f-b8d0-eae805debefd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:32.307048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdf2c4-22b0-4a7f-b8d0-eae805debefd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:32.308504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdf2c4-22b0-4a7f-b8d0-eae805debefd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-aabdf2c4-22b0-4a7f-b8d0-eae805debefd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:27:32.321690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:32.323306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:32.325002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:27:32.325002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:27:32.325002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:27:32.325002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:27:32.325002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:27:32.325002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:27:32.325002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:32.325002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:32.325002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 528/1048] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:27:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:32.329640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdf2c4-22b0-4a7f-b8d0-eae805debefd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:32.411821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 521/1049] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:27:32 2022] POST /identity/v3/endpoints => generated 327 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:27:32.504802 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdf2c4-22b0-4a7f-b8d0-eae805debefd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:32.555962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:32.584042 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb543d-6ff4-457c-82e9-3544b2596d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:32.585051 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb543d-6ff4-457c-82e9-3544b2596d2a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:32.585535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb543d-6ff4-457c-82e9-3544b2596d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:32.586015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb543d-6ff4-457c-82e9-3544b2596d2a None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:32.587296 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb543d-6ff4-457c-82e9-3544b2596d2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:32.587911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb543d-6ff4-457c-82e9-3544b2596d2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:32.591072 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb543d-6ff4-457c-82e9-3544b2596d2a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:32.614539 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 529/1050] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:27:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:32.621975 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf18a9-e12a-4f41-8c9b-1899d7ccd599 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:32.759463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf18a9-e12a-4f41-8c9b-1899d7ccd599 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:32.791292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:32.811024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b300e-4337-4685-8390-30c1eb788b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:32.812039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b300e-4337-4685-8390-30c1eb788b41 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:32.812527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b300e-4337-4685-8390-30c1eb788b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:32.812946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b300e-4337-4685-8390-30c1eb788b41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:32.814307 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b300e-4337-4685-8390-30c1eb788b41 None admin] RBAC: Adding query filter params (role.id=c856bb63a85742ce8066968715c33568, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=58c8502b91ea4ba79e85818727422410) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:32.814808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b300e-4337-4685-8390-30c1eb788b41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:32.817471 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b300e-4337-4685-8390-30c1eb788b41 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:32.829006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 522/1051] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:27:32 2022] GET /identity/v3/role_assignments?user.id=58c8502b91ea4ba79e85818727422410&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=c856bb63a85742ce8066968715c33568 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:34.427457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad8571-bf8e-4740-bbde-a68a7ceb5a09 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:34.428026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad8571-bf8e-4740-bbde-a68a7ceb5a09 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:34.428396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad8571-bf8e-4740-bbde-a68a7ceb5a09 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:34.431064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 530/1052] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:34.438340 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee5863-1ce0-4eed-b6d4-817564591d12 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:34.439169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee5863-1ce0-4eed-b6d4-817564591d12 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:34.439169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee5863-1ce0-4eed-b6d4-817564591d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:34.470353 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-beee5863-1ce0-4eed-b6d4-817564591d12 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:34.472410 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-beee5863-1ce0-4eed-b6d4-817564591d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:34.583927 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 523/1053] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:34.710010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c06ec-ec56-495a-8dd2-fffe113fba07 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:34.710229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c06ec-ec56-495a-8dd2-fffe113fba07 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:34.710463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c06ec-ec56-495a-8dd2-fffe113fba07 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:34.712024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 531/1054] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:34.718800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804efce-d2fe-47b5-9af5-693f376a1e6f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:34.719177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804efce-d2fe-47b5-9af5-693f376a1e6f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:34.719332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804efce-d2fe-47b5-9af5-693f376a1e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:34.739185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e804efce-d2fe-47b5-9af5-693f376a1e6f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:34.740428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e804efce-d2fe-47b5-9af5-693f376a1e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:34.822184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 524/1055] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:27:34.865288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c06ec-ec56-495a-8dd2-fffe113fba07 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:35.032728 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c06ec-ec56-495a-8dd2-fffe113fba07 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:35.095919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cb99c-4e0e-447a-92c5-36a36ee09673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:35.097017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cb99c-4e0e-447a-92c5-36a36ee09673 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:35.097534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cb99c-4e0e-447a-92c5-36a36ee09673 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:35.098027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cb99c-4e0e-447a-92c5-36a36ee09673 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:35.098574 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804efce-d2fe-47b5-9af5-693f376a1e6f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:35.113336 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cb99c-4e0e-447a-92c5-36a36ee09673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-441cb99c-4e0e-447a-92c5-36a36ee09673 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:27:35.119861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:35.121746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:35.123604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:35.123604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:35.123604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:35.123604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:35.123604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:27:35.123604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:27:35.123604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:27:35.123604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:27:35.123604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:27:35.123604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:27:35.123604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:27:35.123604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:35.123604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 532/1056] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:27:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:35.127230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cb99c-4e0e-447a-92c5-36a36ee09673 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:35.180305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:35.183417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:35.262366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cb99c-4e0e-447a-92c5-36a36ee09673 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:35.274377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804efce-d2fe-47b5-9af5-693f376a1e6f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:35.314481 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e047d-adf0-4012-adf4-2d5b197b94d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:35.315903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e047d-adf0-4012-adf4-2d5b197b94d5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:35.316202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e047d-adf0-4012-adf4-2d5b197b94d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:35.316534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e047d-adf0-4012-adf4-2d5b197b94d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:35.317135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e047d-adf0-4012-adf4-2d5b197b94d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:35.317575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e047d-adf0-4012-adf4-2d5b197b94d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:35.318907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e047d-adf0-4012-adf4-2d5b197b94d5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:35.323169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049c959-87d7-4670-9048-ac92ac10c644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:35.323979 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049c959-87d7-4670-9048-ac92ac10c644 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:35.324157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049c959-87d7-4670-9048-ac92ac10c644 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:35.324318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049c959-87d7-4670-9048-ac92ac10c644 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:35.331027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 533/1057] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:27:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:35.332811 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04624b-c3e1-4f1a-a13e-da60bc339840 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:35.333235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04624b-c3e1-4f1a-a13e-da60bc339840 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:35.333638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04624b-c3e1-4f1a-a13e-da60bc339840 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:35.335682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 534/1058] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:35.340207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04624b-c3e1-4f1a-a13e-da60bc339840 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:35.408138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049c959-87d7-4670-9048-ac92ac10c644 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:35.410472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049c959-87d7-4670-9048-ac92ac10c644 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:35.467871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04624b-c3e1-4f1a-a13e-da60bc339840 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:35.475997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:35.477940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:35.506519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe202f5a-0b90-4646-bef6-cb14b3dcaac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:35.507512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe202f5a-0b90-4646-bef6-cb14b3dcaac7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:35.507907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe202f5a-0b90-4646-bef6-cb14b3dcaac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:35.508209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe202f5a-0b90-4646-bef6-cb14b3dcaac7 None admin] PATH_INFO: `/v3/users/b30ade79920b4f249c25c1f4cec69bc6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:35.523598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe202f5a-0b90-4646-bef6-cb14b3dcaac7 None admin] RBAC: Authorizing `identity:get_user(user_id=b30ade79920b4f249c25c1f4cec69bc6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:35.525053 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe202f5a-0b90-4646-bef6-cb14b3dcaac7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:35.526851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 535/1059] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:27:35 2022] GET /identity/v3/users/b30ade79920b4f249c25c1f4cec69bc6 => generated 285 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:35.528068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33016294-5cc4-460f-9443-6a06807bd600 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:35.528341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33016294-5cc4-460f-9443-6a06807bd600 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:35.528602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33016294-5cc4-460f-9443-6a06807bd600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:35.543596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 525/1060] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:27:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:27:35.545248 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049c959-87d7-4670-9048-ac92ac10c644 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:35.550565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-33016294-5cc4-460f-9443-6a06807bd600 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:35.552101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-33016294-5cc4-460f-9443-6a06807bd600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:35.627860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 536/1061] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:35.629019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33016294-5cc4-460f-9443-6a06807bd600 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:35.676883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049c959-87d7-4670-9048-ac92ac10c644 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:35.715677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e495253-b190-4015-b144-4b654fbe7bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:35.716543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e495253-b190-4015-b144-4b654fbe7bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:35.716817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e495253-b190-4015-b144-4b654fbe7bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:35.717091 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e495253-b190-4015-b144-4b654fbe7bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:35.734838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:35.767702 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:35.776364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33016294-5cc4-460f-9443-6a06807bd600 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:35.810501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e495253-b190-4015-b144-4b654fbe7bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:35.813550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e495253-b190-4015-b144-4b654fbe7bd2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:35.817010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a17ed-b48a-4fbe-af19-04bbcde11e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:35.817643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a17ed-b48a-4fbe-af19-04bbcde11e72 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:35.817915 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a17ed-b48a-4fbe-af19-04bbcde11e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:35.818213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a17ed-b48a-4fbe-af19-04bbcde11e72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:35.829663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a17ed-b48a-4fbe-af19-04bbcde11e72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:35.831149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a17ed-b48a-4fbe-af19-04bbcde11e72 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c79a17ed-b48a-4fbe-af19-04bbcde11e72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:27:35.847874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:35.848946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:27:35.850091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:27:35.850091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:35.850091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:35.854253 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 537/1062] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:35.855933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a17ed-b48a-4fbe-af19-04bbcde11e72 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:35.981290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 526/1063] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:27:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:27:35.982064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e495253-b190-4015-b144-4b654fbe7bd2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:35.985615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a17ed-b48a-4fbe-af19-04bbcde11e72 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:36.025380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0f777-c7a7-4bc3-8cf5-4f6d3e78aeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:36.026005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0f777-c7a7-4bc3-8cf5-4f6d3e78aeb4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:36.026384 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0f777-c7a7-4bc3-8cf5-4f6d3e78aeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:36.026645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0f777-c7a7-4bc3-8cf5-4f6d3e78aeb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:36.027293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0f777-c7a7-4bc3-8cf5-4f6d3e78aeb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:36.027590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0f777-c7a7-4bc3-8cf5-4f6d3e78aeb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:36.028980 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0f777-c7a7-4bc3-8cf5-4f6d3e78aeb4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:36.034795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:36.042881 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 538/1064] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:36.043945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0f777-c7a7-4bc3-8cf5-4f6d3e78aeb4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:36.066968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:36.102527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e495253-b190-4015-b144-4b654fbe7bd2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:36.151936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94c655-ef7c-4264-b539-7b9c1e565c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:36.152582 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94c655-ef7c-4264-b539-7b9c1e565c2b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:36.152916 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94c655-ef7c-4264-b539-7b9c1e565c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:36.153170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94c655-ef7c-4264-b539-7b9c1e565c2b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:36.165757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a94c655-ef7c-4264-b539-7b9c1e565c2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3a94c655-ef7c-4264-b539-7b9c1e565c2b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:27:36.171527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:36.172849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:36.174700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:36.174700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:36.174700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:36.174700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:36.174700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:27:36.174700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:27:36.174700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:27:36.174700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:27:36.174700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:27:36.174700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:27:36.174700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 08:27:36.174700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:36.174700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 527/1065] 10.222.0.44 () {56 vars in 1120 bytes} [Sun Sep 11 08:27:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:36.179085 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94c655-ef7c-4264-b539-7b9c1e565c2b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:36.204620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0f777-c7a7-4bc3-8cf5-4f6d3e78aeb4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:36.252519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c11ca-b258-421d-9611-84acf89a5862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:36.253511 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c11ca-b258-421d-9611-84acf89a5862 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:36.253784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c11ca-b258-421d-9611-84acf89a5862 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:36.254138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c11ca-b258-421d-9611-84acf89a5862 None admin] PATH_INFO: `/v3/projects/0f479859e90d4428bc41bc4abbeca8db` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:36.256785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c11ca-b258-421d-9611-84acf89a5862 None admin] RBAC: Authorizing `identity:get_project(project_id=0f479859e90d4428bc41bc4abbeca8db)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:36.259399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c11ca-b258-421d-9611-84acf89a5862 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:36.262391 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 539/1066] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:27:36 2022] GET /identity/v3/projects/0f479859e90d4428bc41bc4abbeca8db => generated 311 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:36.270281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c11ca-b258-421d-9611-84acf89a5862 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:36.322522 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94c655-ef7c-4264-b539-7b9c1e565c2b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:36.331197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:36.368392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc31826-4f81-469d-aab4-d6791d4ef62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:36.369142 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc31826-4f81-469d-aab4-d6791d4ef62b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:36.369684 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc31826-4f81-469d-aab4-d6791d4ef62b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:36.369958 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc31826-4f81-469d-aab4-d6791d4ef62b None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:36.371105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc31826-4f81-469d-aab4-d6791d4ef62b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:36.374185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc31826-4f81-469d-aab4-d6791d4ef62b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:36.374504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc31826-4f81-469d-aab4-d6791d4ef62b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:36.390185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 528/1067] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 08:27:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:36.393678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc31826-4f81-469d-aab4-d6791d4ef62b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:36.396054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:36.437172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c11ca-b258-421d-9611-84acf89a5862 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:36.477299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0abc1-c93c-4fd7-b392-2daea73a9486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:36.477993 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0abc1-c93c-4fd7-b392-2daea73a9486 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:36.478204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0abc1-c93c-4fd7-b392-2daea73a9486 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:36.478478 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0abc1-c93c-4fd7-b392-2daea73a9486 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:36.479209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0abc1-c93c-4fd7-b392-2daea73a9486 None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=0f479859e90d4428bc41bc4abbeca8db, user.id=b30ade79920b4f249c25c1f4cec69bc6) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:36.479514 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0abc1-c93c-4fd7-b392-2daea73a9486 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:36.480980 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0abc1-c93c-4fd7-b392-2daea73a9486 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:36.487569 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 540/1068] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:27:36 2022] GET /identity/v3/role_assignments?user.id=b30ade79920b4f249c25c1f4cec69bc6&scope.project.id=0f479859e90d4428bc41bc4abbeca8db&role.id=020bc02dca71485bab2312442adaa179 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:36.524991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc31826-4f81-469d-aab4-d6791d4ef62b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:36.567827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688f776-be3e-4e85-ab89-0df0231c5ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:36.568489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f776-be3e-4e85-ab89-0df0231c5ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:36.568766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f776-be3e-4e85-ab89-0df0231c5ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:36.569021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688f776-be3e-4e85-ab89-0df0231c5ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:36.582108 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688f776-be3e-4e85-ab89-0df0231c5ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:36.583666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688f776-be3e-4e85-ab89-0df0231c5ed3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1688f776-be3e-4e85-ab89-0df0231c5ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:27:36.601195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:36.603079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:27:36.604933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:27:36.604933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:36.604933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:36.604933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 529/1069] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:36.612543 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0abc1-c93c-4fd7-b392-2daea73a9486 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:36.735070 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0abc1-c93c-4fd7-b392-2daea73a9486 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:36.772590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0090187-4fc8-4b02-89f5-54343b3d9eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:36.773203 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0090187-4fc8-4b02-89f5-54343b3d9eeb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:36.773479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0090187-4fc8-4b02-89f5-54343b3d9eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:36.773752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0090187-4fc8-4b02-89f5-54343b3d9eeb None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:36.774433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0090187-4fc8-4b02-89f5-54343b3d9eeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:36.774733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0090187-4fc8-4b02-89f5-54343b3d9eeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:36.776175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0090187-4fc8-4b02-89f5-54343b3d9eeb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:36.785063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:36.791803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 541/1070] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:36.797852 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688f776-be3e-4e85-ab89-0df0231c5ed3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:36.838056 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:36.967128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688f776-be3e-4e85-ab89-0df0231c5ed3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:37.004001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f51209-276f-47f9-85b8-6ae0d09c5dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:37.004632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f51209-276f-47f9-85b8-6ae0d09c5dcb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:37.004906 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f51209-276f-47f9-85b8-6ae0d09c5dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:37.005177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f51209-276f-47f9-85b8-6ae0d09c5dcb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:37.013600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f51209-276f-47f9-85b8-6ae0d09c5dcb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:37.015624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f51209-276f-47f9-85b8-6ae0d09c5dcb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-a2f51209-276f-47f9-85b8-6ae0d09c5dcb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:27:37.027732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:37.029224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:37.031301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:37.031301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:37.031301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:27:37.031301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:27:37.031301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:27:37.031301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:27:37.031301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:27:37.031301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:27:37.031301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:27:37.031301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:37.031301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 530/1071] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 08:27:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:37.035313 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0090187-4fc8-4b02-89f5-54343b3d9eeb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:37.153823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0090187-4fc8-4b02-89f5-54343b3d9eeb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:37.201174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a834a7-a838-42b3-b8d6-33dc68ff3ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:37.201796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a834a7-a838-42b3-b8d6-33dc68ff3ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:37.202069 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a834a7-a838-42b3-b8d6-33dc68ff3ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:37.202927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a834a7-a838-42b3-b8d6-33dc68ff3ef3 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:37.203681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a834a7-a838-42b3-b8d6-33dc68ff3ef3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:37.203982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a834a7-a838-42b3-b8d6-33dc68ff3ef3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:37.205495 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a834a7-a838-42b3-b8d6-33dc68ff3ef3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:37.227525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 542/1072] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 08:27:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:37.234543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f51209-276f-47f9-85b8-6ae0d09c5dcb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:37.344988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:37.365587 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f51209-276f-47f9-85b8-6ae0d09c5dcb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:37.400607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ddcea-895a-4354-ac10-b5040cfd07c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:37.401263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ddcea-895a-4354-ac10-b5040cfd07c0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:37.401538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ddcea-895a-4354-ac10-b5040cfd07c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:37.401808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ddcea-895a-4354-ac10-b5040cfd07c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:37.414377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ddcea-895a-4354-ac10-b5040cfd07c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:37.415825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ddcea-895a-4354-ac10-b5040cfd07c0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6d6ddcea-895a-4354-ac10-b5040cfd07c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:27:37.431165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:37.432286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:27:37.433286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:27:37.433286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:37.433286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:37.433286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 531/1073] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:37.438846 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a834a7-a838-42b3-b8d6-33dc68ff3ef3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:37.508474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:37.561817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a834a7-a838-42b3-b8d6-33dc68ff3ef3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:37.603277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39fd61-7035-41e0-8559-781f56b23e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:37.603804 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39fd61-7035-41e0-8559-781f56b23e2b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:37.604079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39fd61-7035-41e0-8559-781f56b23e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:37.604347 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39fd61-7035-41e0-8559-781f56b23e2b None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:37.605005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39fd61-7035-41e0-8559-781f56b23e2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:37.605302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39fd61-7035-41e0-8559-781f56b23e2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:37.606776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39fd61-7035-41e0-8559-781f56b23e2b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:37.625662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 543/1074] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:37.627763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ddcea-895a-4354-ac10-b5040cfd07c0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:37.750397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ddcea-895a-4354-ac10-b5040cfd07c0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:37.783874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:37.789696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bbe8f-b5dd-48e5-a772-67538a5074ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:37.790375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bbe8f-b5dd-48e5-a772-67538a5074ac None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:37.790653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bbe8f-b5dd-48e5-a772-67538a5074ac None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:37.803913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bbe8f-b5dd-48e5-a772-67538a5074ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:37.803913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bbe8f-b5dd-48e5-a772-67538a5074ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:37.803913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bbe8f-b5dd-48e5-a772-67538a5074ac None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-346bbe8f-b5dd-48e5-a772-67538a5074ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:27:37.813438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:37.814480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:37.815461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:27:37.815461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:27:37.815461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:27:37.815461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:27:37.815461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:27:37.815461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:27:37.815461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:37.815461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:37.815461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 532/1075] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:27:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:37.820686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39fd61-7035-41e0-8559-781f56b23e2b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:37.953299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39fd61-7035-41e0-8559-781f56b23e2b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:37.986074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:38.025037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeeeccf-7d1d-47ca-ad75-49f829416fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:38.026067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeeeccf-7d1d-47ca-ad75-49f829416fcb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:38.026628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeeeccf-7d1d-47ca-ad75-49f829416fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:38.027116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeeeccf-7d1d-47ca-ad75-49f829416fcb None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:38.028401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeeeccf-7d1d-47ca-ad75-49f829416fcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:38.028953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeeeccf-7d1d-47ca-ad75-49f829416fcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:38.031911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeeeccf-7d1d-47ca-ad75-49f829416fcb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:38.058477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 544/1076] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:27:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:38.060113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bbe8f-b5dd-48e5-a772-67538a5074ac None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:38.170705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bbe8f-b5dd-48e5-a772-67538a5074ac None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:38.207477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad9c57-294e-4ff5-b005-aa920f7a1012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:38.208166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad9c57-294e-4ff5-b005-aa920f7a1012 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:38.208454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad9c57-294e-4ff5-b005-aa920f7a1012 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:38.208727 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad9c57-294e-4ff5-b005-aa920f7a1012 None admin] PATH_INFO: `/v3/projects/811863e9c94d44a39647800f306505f0/users/58c8502b91ea4ba79e85818727422410/roles/c856bb63a85742ce8066968715c33568` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:38.210958 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad9c57-294e-4ff5-b005-aa920f7a1012 None admin] RBAC: Authorizing `identity:create_grant(project_id=811863e9c94d44a39647800f306505f0, user_id=58c8502b91ea4ba79e85818727422410, role_id=c856bb63a85742ce8066968715c33568)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:38.212396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad9c57-294e-4ff5-b005-aa920f7a1012 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:38.289682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 533/1077] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:27:38 2022] PUT /identity/v3/projects/811863e9c94d44a39647800f306505f0/users/58c8502b91ea4ba79e85818727422410/roles/c856bb63a85742ce8066968715c33568 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:27:38.729241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc353ac0-5150-4742-b6e8-17167d20ec82 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:38.729780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc353ac0-5150-4742-b6e8-17167d20ec82 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:38.729780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc353ac0-5150-4742-b6e8-17167d20ec82 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:38.731301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 545/1078] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:38.737481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a95bb-9b64-41c6-9e2c-38fa10dd331b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:38.737926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a95bb-9b64-41c6-9e2c-38fa10dd331b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:38.738037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a95bb-9b64-41c6-9e2c-38fa10dd331b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:38.756279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0b6a95bb-9b64-41c6-9e2c-38fa10dd331b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:38.757480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0b6a95bb-9b64-41c6-9e2c-38fa10dd331b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:38.806170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:38.826091 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 534/1079] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:39.096188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc353ac0-5150-4742-b6e8-17167d20ec82 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:39.201135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc353ac0-5150-4742-b6e8-17167d20ec82 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:39.239466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cff8a5-3d90-4522-8524-5ae2de8c5cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:39.240086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cff8a5-3d90-4522-8524-5ae2de8c5cfd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:39.240357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cff8a5-3d90-4522-8524-5ae2de8c5cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:39.240618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cff8a5-3d90-4522-8524-5ae2de8c5cfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:39.258106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cff8a5-3d90-4522-8524-5ae2de8c5cfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-82cff8a5-3d90-4522-8524-5ae2de8c5cfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:27:39.262953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:39.263985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:39.265433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:39.265433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:39.265433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:39.265433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:39.265433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:27:39.265433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:27:39.265433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:27:39.265433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:27:39.265433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:27:39.265433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:27:39.265433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:27:39.265433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:39.265433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 546/1080] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:27:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:39.270664 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a95bb-9b64-41c6-9e2c-38fa10dd331b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:39.372009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a95bb-9b64-41c6-9e2c-38fa10dd331b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:39.407386 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e48b7-f2f7-4bb9-b06d-2773affaac3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:39.407995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e48b7-f2f7-4bb9-b06d-2773affaac3b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:39.408271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e48b7-f2f7-4bb9-b06d-2773affaac3b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:39.408537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e48b7-f2f7-4bb9-b06d-2773affaac3b None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:39.409203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e48b7-f2f7-4bb9-b06d-2773affaac3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:39.409505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e48b7-f2f7-4bb9-b06d-2773affaac3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:39.410903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e48b7-f2f7-4bb9-b06d-2773affaac3b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:39.424045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 535/1081] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:27:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:39.425304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cff8a5-3d90-4522-8524-5ae2de8c5cfd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:39.460718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:39.525359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cff8a5-3d90-4522-8524-5ae2de8c5cfd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:39.559713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c49ef8-f593-48f6-bd96-34e9d9a07df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:39.560361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c49ef8-f593-48f6-bd96-34e9d9a07df7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:39.560632 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c49ef8-f593-48f6-bd96-34e9d9a07df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:39.560897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c49ef8-f593-48f6-bd96-34e9d9a07df7 None admin] PATH_INFO: `/v3/users/b30ade79920b4f249c25c1f4cec69bc6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:39.562672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c49ef8-f593-48f6-bd96-34e9d9a07df7 None admin] RBAC: Authorizing `identity:get_user(user_id=b30ade79920b4f249c25c1f4cec69bc6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:39.564141 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c49ef8-f593-48f6-bd96-34e9d9a07df7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:39.565867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 547/1082] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:27:39 2022] GET /identity/v3/users/b30ade79920b4f249c25c1f4cec69bc6 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:39.572255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e48b7-f2f7-4bb9-b06d-2773affaac3b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:39.619573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:39.673651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e48b7-f2f7-4bb9-b06d-2773affaac3b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:39.709450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a80ff-fa23-45dd-9812-da053cc88a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:39.710076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a80ff-fa23-45dd-9812-da053cc88a8a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:39.710384 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a80ff-fa23-45dd-9812-da053cc88a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:39.710686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a80ff-fa23-45dd-9812-da053cc88a8a None admin] PATH_INFO: `/v3/projects/0f479859e90d4428bc41bc4abbeca8db` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:39.711752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a80ff-fa23-45dd-9812-da053cc88a8a None admin] RBAC: Authorizing `identity:get_project(project_id=0f479859e90d4428bc41bc4abbeca8db)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:39.713167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a80ff-fa23-45dd-9812-da053cc88a8a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:39.715018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 536/1083] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:27:39 2022] GET /identity/v3/projects/0f479859e90d4428bc41bc4abbeca8db => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:39.721225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c49ef8-f593-48f6-bd96-34e9d9a07df7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:39.824880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c49ef8-f593-48f6-bd96-34e9d9a07df7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:39.827366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:39.864433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e8203-b4e2-4624-a3ff-6d21fbbd4703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:39.865095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e8203-b4e2-4624-a3ff-6d21fbbd4703 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:39.865372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e8203-b4e2-4624-a3ff-6d21fbbd4703 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:39.865641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e8203-b4e2-4624-a3ff-6d21fbbd4703 None admin] PATH_INFO: `/v3/projects/0f479859e90d4428bc41bc4abbeca8db/users/b30ade79920b4f249c25c1f4cec69bc6/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:39.867590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e8203-b4e2-4624-a3ff-6d21fbbd4703 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f479859e90d4428bc41bc4abbeca8db, user_id=b30ade79920b4f249c25c1f4cec69bc6, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:39.869031 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e8203-b4e2-4624-a3ff-6d21fbbd4703 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:39.941925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 548/1084] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:27:39 2022] PUT /identity/v3/projects/0f479859e90d4428bc41bc4abbeca8db/users/b30ade79920b4f249c25c1f4cec69bc6/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:27:40.519541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcddcb-2afc-40d7-8430-700f34b08b06 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:40.520157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcddcb-2afc-40d7-8430-700f34b08b06 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:40.520157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcddcb-2afc-40d7-8430-700f34b08b06 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:40.521424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 537/1085] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:40.527551 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb28889-d433-4814-979c-c1300748f100 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:40.527900 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb28889-d433-4814-979c-c1300748f100 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:40.528174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb28889-d433-4814-979c-c1300748f100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:40.553215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bbb28889-d433-4814-979c-c1300748f100 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:40.554455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bbb28889-d433-4814-979c-c1300748f100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:40.629283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 549/1086] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:40.940144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcddcb-2afc-40d7-8430-700f34b08b06 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:41.093007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcddcb-2afc-40d7-8430-700f34b08b06 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:41.106918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:41.133369 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56328e35-a14c-46ef-942e-3e4c3b913509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:41.134168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56328e35-a14c-46ef-942e-3e4c3b913509 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:41.134465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56328e35-a14c-46ef-942e-3e4c3b913509 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:41.134867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56328e35-a14c-46ef-942e-3e4c3b913509 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:41.147590 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56328e35-a14c-46ef-942e-3e4c3b913509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-56328e35-a14c-46ef-942e-3e4c3b913509 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:27:41.152767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:41.154021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:41.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:41.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:41.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:41.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:41.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:27:41.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:27:41.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:27:41.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:27:41.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:27:41.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:27:41.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 08:27:41.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:41.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 538/1087] 10.222.0.44 () {56 vars in 1120 bytes} [Sun Sep 11 08:27:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:41.160884 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb28889-d433-4814-979c-c1300748f100 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:41.284965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:41.292745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb28889-d433-4814-979c-c1300748f100 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:41.334002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2489c3-3210-46df-82ce-31d64dc15003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:41.334661 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2489c3-3210-46df-82ce-31d64dc15003 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:41.334931 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2489c3-3210-46df-82ce-31d64dc15003 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:41.335201 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2489c3-3210-46df-82ce-31d64dc15003 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:41.335884 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2489c3-3210-46df-82ce-31d64dc15003 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:41.336183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2489c3-3210-46df-82ce-31d64dc15003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:41.337597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2489c3-3210-46df-82ce-31d64dc15003 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:41.348448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 550/1088] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 08:27:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:41.354879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56328e35-a14c-46ef-942e-3e4c3b913509 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:41.467164 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56328e35-a14c-46ef-942e-3e4c3b913509 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:41.502913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9a10a-dccb-4ad1-9c90-ef504dfb8839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:41.503533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9a10a-dccb-4ad1-9c90-ef504dfb8839 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:41.503836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9a10a-dccb-4ad1-9c90-ef504dfb8839 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:41.504103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9a10a-dccb-4ad1-9c90-ef504dfb8839 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:41.514966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9a10a-dccb-4ad1-9c90-ef504dfb8839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:41.516397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9a10a-dccb-4ad1-9c90-ef504dfb8839 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6cb9a10a-dccb-4ad1-9c90-ef504dfb8839 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:27:41.530874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:41.532311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:27:41.533209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:27:41.533209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:41.533209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:41.533209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 539/1089] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:41.537803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2489c3-3210-46df-82ce-31d64dc15003 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:41.642642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2489c3-3210-46df-82ce-31d64dc15003 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:41.678887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43face-7fc2-4a87-8749-2c7aff5cdb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:41.679490 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43face-7fc2-4a87-8749-2c7aff5cdb70 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:41.679760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43face-7fc2-4a87-8749-2c7aff5cdb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:41.680028 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43face-7fc2-4a87-8749-2c7aff5cdb70 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:41.680670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43face-7fc2-4a87-8749-2c7aff5cdb70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:41.680966 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43face-7fc2-4a87-8749-2c7aff5cdb70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:41.682375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43face-7fc2-4a87-8749-2c7aff5cdb70 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:41.695163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 551/1090] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:41.701560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9a10a-dccb-4ad1-9c90-ef504dfb8839 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:41.729662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:41.810061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9a10a-dccb-4ad1-9c90-ef504dfb8839 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:41.844923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab8307-c932-49fe-89df-252f1a682bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:41.845559 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab8307-c932-49fe-89df-252f1a682bba None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:41.845837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab8307-c932-49fe-89df-252f1a682bba None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:41.846105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab8307-c932-49fe-89df-252f1a682bba None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:41.854116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab8307-c932-49fe-89df-252f1a682bba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:41.855580 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab8307-c932-49fe-89df-252f1a682bba None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-80ab8307-c932-49fe-89df-252f1a682bba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:27:41.877539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:41.878617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:41.879577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:41.879577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:41.879577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:27:41.879577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:27:41.879577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:27:41.879577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:27:41.879577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:27:41.879577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:27:41.879577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 08:27:41.879577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:41.879577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 540/1091] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 08:27:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:27:41.885616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43face-7fc2-4a87-8749-2c7aff5cdb70 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:41.904630 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:42.004381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43face-7fc2-4a87-8749-2c7aff5cdb70 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:42.043368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1796d5b-5d1c-4b15-99e0-d5f67eefc589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:42.043985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1796d5b-5d1c-4b15-99e0-d5f67eefc589 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:42.044256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1796d5b-5d1c-4b15-99e0-d5f67eefc589 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:42.044522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1796d5b-5d1c-4b15-99e0-d5f67eefc589 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:42.045261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1796d5b-5d1c-4b15-99e0-d5f67eefc589 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:42.045557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1796d5b-5d1c-4b15-99e0-d5f67eefc589 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:42.047106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1796d5b-5d1c-4b15-99e0-d5f67eefc589 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:42.066672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 552/1092] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 08:27:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:42.072595 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab8307-c932-49fe-89df-252f1a682bba None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:42.175367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:42.192798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab8307-c932-49fe-89df-252f1a682bba None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:42.221736 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531418c-c5db-4038-a6d9-d8e121523963 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:42.222375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531418c-c5db-4038-a6d9-d8e121523963 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:42.222375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531418c-c5db-4038-a6d9-d8e121523963 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:42.223647 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 553/1093] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:42.229092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d59712-b72c-4174-9e8d-9e9ccb08399d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:42.229372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d59712-b72c-4174-9e8d-9e9ccb08399d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:42.229644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d59712-b72c-4174-9e8d-9e9ccb08399d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:42.232109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851640d-e3af-4d46-a249-63a22a5ee133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:42.232719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851640d-e3af-4d46-a249-63a22a5ee133 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:42.232992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851640d-e3af-4d46-a249-63a22a5ee133 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:42.233257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851640d-e3af-4d46-a249-63a22a5ee133 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:42.251902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-48d59712-b72c-4174-9e8d-9e9ccb08399d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:42.253198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-48d59712-b72c-4174-9e8d-9e9ccb08399d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:42.317842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851640d-e3af-4d46-a249-63a22a5ee133 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:42.319337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851640d-e3af-4d46-a249-63a22a5ee133 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:42.326646 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 554/1094] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:27:42.420200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:42.439995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 541/1095] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:27:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:27:42.445909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d59712-b72c-4174-9e8d-9e9ccb08399d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:42.450292 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:42.561935 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d59712-b72c-4174-9e8d-9e9ccb08399d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:42.598465 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507c35e-af34-4c51-9580-8fdae6f8929f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:42.598985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507c35e-af34-4c51-9580-8fdae6f8929f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:42.599255 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507c35e-af34-4c51-9580-8fdae6f8929f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:42.599522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507c35e-af34-4c51-9580-8fdae6f8929f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:42.605296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851640d-e3af-4d46-a249-63a22a5ee133 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:42.610612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507c35e-af34-4c51-9580-8fdae6f8929f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:42.612096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507c35e-af34-4c51-9580-8fdae6f8929f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c507c35e-af34-4c51-9580-8fdae6f8929f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:27:42.635954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:42.637048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:27:42.638124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:27:42.638124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:27:42.638124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:42.638124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 555/1096] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:42.647035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507c35e-af34-4c51-9580-8fdae6f8929f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:42.712689 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:42.734859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851640d-e3af-4d46-a249-63a22a5ee133 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:42.766801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507c35e-af34-4c51-9580-8fdae6f8929f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:42.778077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d7bff-b12f-4f53-854a-5ce3992810e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:42.779245 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d7bff-b12f-4f53-854a-5ce3992810e1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:42.779718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d7bff-b12f-4f53-854a-5ce3992810e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:42.780148 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d7bff-b12f-4f53-854a-5ce3992810e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:42.797068 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d7bff-b12f-4f53-854a-5ce3992810e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-a32d7bff-b12f-4f53-854a-5ce3992810e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:27:42.801747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:42.802926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:42.804089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:42.804089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:42.804089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:42.804089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:42.804089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:27:42.804089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:27:42.804089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:27:42.804089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:27:42.804089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:27:42.804089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:27:42.804089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:27:42.804089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:27:42.804089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 542/1097] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:27:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:42.809514 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d7bff-b12f-4f53-854a-5ce3992810e1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:42.817133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ad6f4-1b25-4520-a5c6-d38ccf496e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:42.817962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ad6f4-1b25-4520-a5c6-d38ccf496e25 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:42.818266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ad6f4-1b25-4520-a5c6-d38ccf496e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:42.818532 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ad6f4-1b25-4520-a5c6-d38ccf496e25 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:42.819179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ad6f4-1b25-4520-a5c6-d38ccf496e25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:42.819479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ad6f4-1b25-4520-a5c6-d38ccf496e25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:42.821275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ad6f4-1b25-4520-a5c6-d38ccf496e25 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:42.834268 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 556/1098] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:42.840004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ad6f4-1b25-4520-a5c6-d38ccf496e25 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:42.866694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:42.934115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d7bff-b12f-4f53-854a-5ce3992810e1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:42.952737 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ad6f4-1b25-4520-a5c6-d38ccf496e25 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:42.969402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:42.970499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ae0cb-d1d0-4909-8de8-ca12c90033cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:42.971110 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ae0cb-d1d0-4909-8de8-ca12c90033cc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:42.971382 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ae0cb-d1d0-4909-8de8-ca12c90033cc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:42.971652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ae0cb-d1d0-4909-8de8-ca12c90033cc None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:42.972311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ae0cb-d1d0-4909-8de8-ca12c90033cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:42.972612 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ae0cb-d1d0-4909-8de8-ca12c90033cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:42.973968 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ae0cb-d1d0-4909-8de8-ca12c90033cc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:42.982127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 543/1099] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:27:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:42.987758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55213f0f-fb0f-4777-9fe3-a6113fc211b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:42.996414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55213f0f-fb0f-4777-9fe3-a6113fc211b5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:42.996567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ae0cb-d1d0-4909-8de8-ca12c90033cc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:42.996664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55213f0f-fb0f-4777-9fe3-a6113fc211b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:42.996664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55213f0f-fb0f-4777-9fe3-a6113fc211b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:42.996664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55213f0f-fb0f-4777-9fe3-a6113fc211b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:42.997307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55213f0f-fb0f-4777-9fe3-a6113fc211b5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-55213f0f-fb0f-4777-9fe3-a6113fc211b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:27:43.009224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:27:43.010395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:27:43.011343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:27:43.011343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:27:43.011343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:27:43.011343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:27:43.011343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:27:43.011343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:27:43.011343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:27:43.011343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:27:43.011343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 557/1100] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:27:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:27:43.016195 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55213f0f-fb0f-4777-9fe3-a6113fc211b5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:43.101869 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ae0cb-d1d0-4909-8de8-ca12c90033cc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:43.111245 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:43.124032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55213f0f-fb0f-4777-9fe3-a6113fc211b5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:43.144283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991ca34-aef9-40b2-bd8a-23fc540d2d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:43.144926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991ca34-aef9-40b2-bd8a-23fc540d2d4b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:43.145202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991ca34-aef9-40b2-bd8a-23fc540d2d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:43.145503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991ca34-aef9-40b2-bd8a-23fc540d2d4b None admin] PATH_INFO: `/v3/users/b30ade79920b4f249c25c1f4cec69bc6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:43.146832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991ca34-aef9-40b2-bd8a-23fc540d2d4b None admin] RBAC: Authorizing `identity:get_user(user_id=b30ade79920b4f249c25c1f4cec69bc6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:43.148552 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991ca34-aef9-40b2-bd8a-23fc540d2d4b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:43.150213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 544/1101] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:27:42 2022] GET /identity/v3/users/b30ade79920b4f249c25c1f4cec69bc6 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:43.156411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991ca34-aef9-40b2-bd8a-23fc540d2d4b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:43.158604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924e4f4-417a-4301-8444-9b6b23a164f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:43.159206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924e4f4-417a-4301-8444-9b6b23a164f0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:43.159480 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924e4f4-417a-4301-8444-9b6b23a164f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:43.159747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924e4f4-417a-4301-8444-9b6b23a164f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:43.160434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924e4f4-417a-4301-8444-9b6b23a164f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:43.160734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924e4f4-417a-4301-8444-9b6b23a164f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:43.162241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924e4f4-417a-4301-8444-9b6b23a164f0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:43.175068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 558/1102] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:27:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:43.181161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924e4f4-417a-4301-8444-9b6b23a164f0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:43.215224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:43.257162 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991ca34-aef9-40b2-bd8a-23fc540d2d4b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:43.279725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924e4f4-417a-4301-8444-9b6b23a164f0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:43.301260 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ceea3-c873-48ae-abd9-fde02caced25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:43.301877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ceea3-c873-48ae-abd9-fde02caced25 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:43.302173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ceea3-c873-48ae-abd9-fde02caced25 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:43.302455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ceea3-c873-48ae-abd9-fde02caced25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:43.317094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93f122-329d-47bc-a332-45901ddc1c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:43.317869 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f122-329d-47bc-a332-45901ddc1c29 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:43.318600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f122-329d-47bc-a332-45901ddc1c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:43.318874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f122-329d-47bc-a332-45901ddc1c29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:43.319620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93f122-329d-47bc-a332-45901ddc1c29 None admin] RBAC: Adding query filter params (role.id=c856bb63a85742ce8066968715c33568, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=58c8502b91ea4ba79e85818727422410) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:43.319925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93f122-329d-47bc-a332-45901ddc1c29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:43.321398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93f122-329d-47bc-a332-45901ddc1c29 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:43.327980 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 559/1103] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:27:43 2022] GET /identity/v3/role_assignments?user.id=58c8502b91ea4ba79e85818727422410&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=c856bb63a85742ce8066968715c33568 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:27:43.350853 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:43.378969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ceea3-c873-48ae-abd9-fde02caced25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:43.380344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ceea3-c873-48ae-abd9-fde02caced25 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:43.494358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 545/1104] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:27:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 08:27:43.500654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93f122-329d-47bc-a332-45901ddc1c29 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:43.605175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:43.608101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93f122-329d-47bc-a332-45901ddc1c29 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:43.642137 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e646b0-c640-42ad-97e5-72e00c114109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:43.642724 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e646b0-c640-42ad-97e5-72e00c114109 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:43.642995 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e646b0-c640-42ad-97e5-72e00c114109 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:43.643266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e646b0-c640-42ad-97e5-72e00c114109 None admin] PATH_INFO: `/v3/projects/0f479859e90d4428bc41bc4abbeca8db` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:43.644368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e646b0-c640-42ad-97e5-72e00c114109 None admin] RBAC: Authorizing `identity:get_project(project_id=0f479859e90d4428bc41bc4abbeca8db)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:43.645787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e646b0-c640-42ad-97e5-72e00c114109 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:43.647427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 560/1105] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:27:43 2022] GET /identity/v3/projects/0f479859e90d4428bc41bc4abbeca8db => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:43.653641 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ceea3-c873-48ae-abd9-fde02caced25 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:43.733395 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:43.757724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ceea3-c873-48ae-abd9-fde02caced25 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:43.795377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1c935-3b27-4d86-bb16-ed5bb1fc4823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:43.796044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1c935-3b27-4d86-bb16-ed5bb1fc4823 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:43.796315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1c935-3b27-4d86-bb16-ed5bb1fc4823 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:43.796584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1c935-3b27-4d86-bb16-ed5bb1fc4823 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:43.797294 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1c935-3b27-4d86-bb16-ed5bb1fc4823 None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=0f479859e90d4428bc41bc4abbeca8db, user.id=b30ade79920b4f249c25c1f4cec69bc6) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:43.797596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1c935-3b27-4d86-bb16-ed5bb1fc4823 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:43.799037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1c935-3b27-4d86-bb16-ed5bb1fc4823 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:43.804867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 546/1106] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:27:43 2022] GET /identity/v3/role_assignments?user.id=b30ade79920b4f249c25c1f4cec69bc6&scope.project.id=0f479859e90d4428bc41bc4abbeca8db&role.id=020bc02dca71485bab2312442adaa179 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:46.007900 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e1984-95f2-425a-8f0c-9a57b2d124df None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:46.008452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e1984-95f2-425a-8f0c-9a57b2d124df None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:46.008452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e1984-95f2-425a-8f0c-9a57b2d124df None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:46.009750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 561/1107] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:46.015543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0632f7-d33c-4c18-9fdd-f57f49542926 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:46.015953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0632f7-d33c-4c18-9fdd-f57f49542926 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:46.016105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0632f7-d33c-4c18-9fdd-f57f49542926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:46.034515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ac0632f7-d33c-4c18-9fdd-f57f49542926 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:46.035707 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ac0632f7-d33c-4c18-9fdd-f57f49542926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:46.099554 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 547/1108] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:46.369142 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e1984-95f2-425a-8f0c-9a57b2d124df None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:46.477309 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e1984-95f2-425a-8f0c-9a57b2d124df None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:46.512412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe22b47-5658-4fe1-b606-1d5f94a94bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:46.513032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe22b47-5658-4fe1-b606-1d5f94a94bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:46.513302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe22b47-5658-4fe1-b606-1d5f94a94bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:46.513566 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe22b47-5658-4fe1-b606-1d5f94a94bb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:46.514720 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe22b47-5658-4fe1-b606-1d5f94a94bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:46.516158 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe22b47-5658-4fe1-b606-1d5f94a94bb9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:46.517707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 562/1109] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:46.523658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0632f7-d33c-4c18-9fdd-f57f49542926 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:46.590124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:46.623194 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0632f7-d33c-4c18-9fdd-f57f49542926 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:46.656894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add70d3b-1239-4c91-acb5-af183063f7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:46.657496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add70d3b-1239-4c91-acb5-af183063f7b5 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:46.657771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add70d3b-1239-4c91-acb5-af183063f7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:46.658037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add70d3b-1239-4c91-acb5-af183063f7b5 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:46.658727 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add70d3b-1239-4c91-acb5-af183063f7b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:46.660144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add70d3b-1239-4c91-acb5-af183063f7b5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:47.117157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 548/1110] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:27:46 2022] POST /identity/v3/users => generated 286 bytes in 594 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:27:49.307139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9a1df-ade2-4475-856c-a6d9772ecb17 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:49.307652 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9a1df-ade2-4475-856c-a6d9772ecb17 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:49.307652 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9a1df-ade2-4475-856c-a6d9772ecb17 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:49.308979 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 563/1111] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:49.314649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8907-742c-4529-a9ce-306a8d58a744 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:49.315059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8907-742c-4529-a9ce-306a8d58a744 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:49.315194 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8907-742c-4529-a9ce-306a8d58a744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:49.333404 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-78cc8907-742c-4529-a9ce-306a8d58a744 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:49.334614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-78cc8907-742c-4529-a9ce-306a8d58a744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:49.402674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 549/1112] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:49.670104 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9a1df-ade2-4475-856c-a6d9772ecb17 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:49.710897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:49.774673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9a1df-ade2-4475-856c-a6d9772ecb17 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:49.808188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec73a05-d429-4002-b59d-cc6189e377b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:49.808814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec73a05-d429-4002-b59d-cc6189e377b8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:49.809094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec73a05-d429-4002-b59d-cc6189e377b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:49.809367 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec73a05-d429-4002-b59d-cc6189e377b8 None admin] PATH_INFO: `/v3/roles/65f33fe3045c433b9a3e045e7572aed6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:49.810770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec73a05-d429-4002-b59d-cc6189e377b8 None admin] RBAC: Authorizing `identity:get_role(role_id=65f33fe3045c433b9a3e045e7572aed6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:49.812679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec73a05-d429-4002-b59d-cc6189e377b8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:49.814005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 564/1113] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:27:49 2022] GET /identity/v3/roles/65f33fe3045c433b9a3e045e7572aed6 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:49.819825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc8907-742c-4529-a9ce-306a8d58a744 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:49.872371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:49.916170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc8907-742c-4529-a9ce-306a8d58a744 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:49.952528 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77fc4a-aaa2-4d0a-a3ea-72d188931337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:49.953095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77fc4a-aaa2-4d0a-a3ea-72d188931337 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:49.953373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77fc4a-aaa2-4d0a-a3ea-72d188931337 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:49.953641 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77fc4a-aaa2-4d0a-a3ea-72d188931337 None admin] PATH_INFO: `/v3/users/c472a30b303b421f9ffa689bdb4e3232` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:49.968862 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77fc4a-aaa2-4d0a-a3ea-72d188931337 None admin] RBAC: Authorizing `identity:get_user(user_id=c472a30b303b421f9ffa689bdb4e3232)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:49.970338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77fc4a-aaa2-4d0a-a3ea-72d188931337 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:49.972059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 550/1114] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:27:49 2022] GET /identity/v3/users/c472a30b303b421f9ffa689bdb4e3232 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:49.978002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec73a05-d429-4002-b59d-cc6189e377b8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:50.078544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec73a05-d429-4002-b59d-cc6189e377b8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:50.080863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:50.112858 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f8413-b87d-4188-99ec-6904d0495112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:50.113450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f8413-b87d-4188-99ec-6904d0495112 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:50.113722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f8413-b87d-4188-99ec-6904d0495112 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:50.113986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f8413-b87d-4188-99ec-6904d0495112 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:50.181261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f8413-b87d-4188-99ec-6904d0495112 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:50.182726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f8413-b87d-4188-99ec-6904d0495112 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:50.276083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 565/1115] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:27:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:27:50.282846 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77fc4a-aaa2-4d0a-a3ea-72d188931337 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:50.378576 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77fc4a-aaa2-4d0a-a3ea-72d188931337 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:50.391603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:50.412352 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7653f9-edc4-4552-b606-b56380cfcc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:50.412985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7653f9-edc4-4552-b606-b56380cfcc1c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:50.413455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7653f9-edc4-4552-b606-b56380cfcc1c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:50.413771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7653f9-edc4-4552-b606-b56380cfcc1c None admin] PATH_INFO: `/v3/projects/0f479859e90d4428bc41bc4abbeca8db` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:50.415301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7653f9-edc4-4552-b606-b56380cfcc1c None admin] RBAC: Authorizing `identity:get_project(project_id=0f479859e90d4428bc41bc4abbeca8db)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:50.416798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7653f9-edc4-4552-b606-b56380cfcc1c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:50.418484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 551/1116] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:27:50 2022] GET /identity/v3/projects/0f479859e90d4428bc41bc4abbeca8db => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:50.424650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f8413-b87d-4188-99ec-6904d0495112 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:50.427755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:50.523716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f8413-b87d-4188-99ec-6904d0495112 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:50.557779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44823d-0ab8-4d56-a76e-2bd1666c47ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:50.558403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44823d-0ab8-4d56-a76e-2bd1666c47ca None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:50.558678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44823d-0ab8-4d56-a76e-2bd1666c47ca None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:50.558945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44823d-0ab8-4d56-a76e-2bd1666c47ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:50.559653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44823d-0ab8-4d56-a76e-2bd1666c47ca None admin] RBAC: Adding query filter params (role.id=65f33fe3045c433b9a3e045e7572aed6, scope.project.id=0f479859e90d4428bc41bc4abbeca8db, user.id=c472a30b303b421f9ffa689bdb4e3232) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:50.559954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44823d-0ab8-4d56-a76e-2bd1666c47ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:50.561897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44823d-0ab8-4d56-a76e-2bd1666c47ca None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:50.567458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 566/1117] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:27:50 2022] GET /identity/v3/role_assignments?user.id=c472a30b303b421f9ffa689bdb4e3232&scope.project.id=0f479859e90d4428bc41bc4abbeca8db&role.id=65f33fe3045c433b9a3e045e7572aed6 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:52.701446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b4f95-5371-4eb3-9e72-2b191434c636 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:52.702552 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b4f95-5371-4eb3-9e72-2b191434c636 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:52.702645 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b4f95-5371-4eb3-9e72-2b191434c636 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:52.703933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 552/1118] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:52.709229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3223b4-4e79-4358-b275-1b91fe9263f7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:52.709673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3223b4-4e79-4358-b275-1b91fe9263f7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:52.709765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3223b4-4e79-4358-b275-1b91fe9263f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:52.728300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-dd3223b4-4e79-4358-b275-1b91fe9263f7 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:52.729472 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dd3223b4-4e79-4358-b275-1b91fe9263f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:52.788898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:52.794296 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 567/1119] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:53.061845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b4f95-5371-4eb3-9e72-2b191434c636 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:53.167961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b4f95-5371-4eb3-9e72-2b191434c636 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:53.201192 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7877ab-7314-45ec-8986-7eb8f05a0456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:53.202391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7877ab-7314-45ec-8986-7eb8f05a0456 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:53.202670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7877ab-7314-45ec-8986-7eb8f05a0456 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:53.202936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7877ab-7314-45ec-8986-7eb8f05a0456 None admin] PATH_INFO: `/v3/roles/65f33fe3045c433b9a3e045e7572aed6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:53.204312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7877ab-7314-45ec-8986-7eb8f05a0456 None admin] RBAC: Authorizing `identity:get_role(role_id=65f33fe3045c433b9a3e045e7572aed6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:53.205669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7877ab-7314-45ec-8986-7eb8f05a0456 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:53.207032 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 553/1120] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:27:53 2022] GET /identity/v3/roles/65f33fe3045c433b9a3e045e7572aed6 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:53.212903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3223b4-4e79-4358-b275-1b91fe9263f7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:53.311794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3223b4-4e79-4358-b275-1b91fe9263f7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:53.347243 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2a802-0114-44ce-a044-a27ed5fade4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:53.347793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2a802-0114-44ce-a044-a27ed5fade4e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:53.348073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2a802-0114-44ce-a044-a27ed5fade4e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:53.348346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2a802-0114-44ce-a044-a27ed5fade4e None admin] PATH_INFO: `/v3/users/c472a30b303b421f9ffa689bdb4e3232` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:53.349463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2a802-0114-44ce-a044-a27ed5fade4e None admin] RBAC: Authorizing `identity:get_user(user_id=c472a30b303b421f9ffa689bdb4e3232)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:53.350929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2a802-0114-44ce-a044-a27ed5fade4e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:53.352514 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 568/1121] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:27:53 2022] GET /identity/v3/users/c472a30b303b421f9ffa689bdb4e3232 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:53.358971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7877ab-7314-45ec-8986-7eb8f05a0456 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:53.407640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:53.456992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7877ab-7314-45ec-8986-7eb8f05a0456 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:53.489767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a7374-3ef1-4d35-967d-3e60bc024416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:53.490410 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a7374-3ef1-4d35-967d-3e60bc024416 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:53.490687 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a7374-3ef1-4d35-967d-3e60bc024416 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:53.490957 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a7374-3ef1-4d35-967d-3e60bc024416 None admin] PATH_INFO: `/v3/projects/0f479859e90d4428bc41bc4abbeca8db` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:53.492051 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a7374-3ef1-4d35-967d-3e60bc024416 None admin] RBAC: Authorizing `identity:get_project(project_id=0f479859e90d4428bc41bc4abbeca8db)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:53.493463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a7374-3ef1-4d35-967d-3e60bc024416 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:53.495111 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 554/1122] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:27:53 2022] GET /identity/v3/projects/0f479859e90d4428bc41bc4abbeca8db => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:53.501530 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2a802-0114-44ce-a044-a27ed5fade4e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:53.567227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:53.600776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2a802-0114-44ce-a044-a27ed5fade4e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:53.633921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e9c01-04e8-48e8-bcfe-1d08186d2538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:53.634614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e9c01-04e8-48e8-bcfe-1d08186d2538 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:53.634892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e9c01-04e8-48e8-bcfe-1d08186d2538 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:53.635162 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e9c01-04e8-48e8-bcfe-1d08186d2538 None admin] PATH_INFO: `/v3/projects/0f479859e90d4428bc41bc4abbeca8db/users/c472a30b303b421f9ffa689bdb4e3232/roles/65f33fe3045c433b9a3e045e7572aed6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:53.637278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e9c01-04e8-48e8-bcfe-1d08186d2538 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f479859e90d4428bc41bc4abbeca8db, user_id=c472a30b303b421f9ffa689bdb4e3232, role_id=65f33fe3045c433b9a3e045e7572aed6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:53.638748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e9c01-04e8-48e8-bcfe-1d08186d2538 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:53.709312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 569/1123] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:27:53 2022] PUT /identity/v3/projects/0f479859e90d4428bc41bc4abbeca8db/users/c472a30b303b421f9ffa689bdb4e3232/roles/65f33fe3045c433b9a3e045e7572aed6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:27:55.850292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc965f75-8ed1-449f-a971-f128912e849b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:55.850824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc965f75-8ed1-449f-a971-f128912e849b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:55.850824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc965f75-8ed1-449f-a971-f128912e849b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:55.852171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 555/1124] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:55.857303 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b826d7a-329c-4e50-9e0f-5d4a459707ed None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:55.857734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b826d7a-329c-4e50-9e0f-5d4a459707ed None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:55.857932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b826d7a-329c-4e50-9e0f-5d4a459707ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:55.876301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6b826d7a-329c-4e50-9e0f-5d4a459707ed None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:55.877481 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6b826d7a-329c-4e50-9e0f-5d4a459707ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:55.941421 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 570/1125] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:56.206751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc965f75-8ed1-449f-a971-f128912e849b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:56.305535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc965f75-8ed1-449f-a971-f128912e849b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:56.321150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:56.338509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631a77d-0db6-4f25-82a5-7c30bee8ebb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:56.339127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631a77d-0db6-4f25-82a5-7c30bee8ebb8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:56.339397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631a77d-0db6-4f25-82a5-7c30bee8ebb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:56.339659 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631a77d-0db6-4f25-82a5-7c30bee8ebb8 None admin] PATH_INFO: `/v3/roles/65f33fe3045c433b9a3e045e7572aed6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:56.341010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631a77d-0db6-4f25-82a5-7c30bee8ebb8 None admin] RBAC: Authorizing `identity:get_role(role_id=65f33fe3045c433b9a3e045e7572aed6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:56.343363 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631a77d-0db6-4f25-82a5-7c30bee8ebb8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:56.344687 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 556/1126] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:27:56 2022] GET /identity/v3/roles/65f33fe3045c433b9a3e045e7572aed6 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:56.350546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b826d7a-329c-4e50-9e0f-5d4a459707ed None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:56.403696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:56.445991 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b826d7a-329c-4e50-9e0f-5d4a459707ed None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:56.480421 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb2eed-4db7-4194-8b24-ae0e5b47a521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:56.480963 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb2eed-4db7-4194-8b24-ae0e5b47a521 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:56.481235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb2eed-4db7-4194-8b24-ae0e5b47a521 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:56.481500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb2eed-4db7-4194-8b24-ae0e5b47a521 None admin] PATH_INFO: `/v3/users/c472a30b303b421f9ffa689bdb4e3232` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:56.482616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb2eed-4db7-4194-8b24-ae0e5b47a521 None admin] RBAC: Authorizing `identity:get_user(user_id=c472a30b303b421f9ffa689bdb4e3232)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:56.484078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb2eed-4db7-4194-8b24-ae0e5b47a521 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:56.485649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 571/1127] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:27:56 2022] GET /identity/v3/users/c472a30b303b421f9ffa689bdb4e3232 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:56.492623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631a77d-0db6-4f25-82a5-7c30bee8ebb8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:56.596018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631a77d-0db6-4f25-82a5-7c30bee8ebb8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:56.631184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c8feb-9ac0-4c03-815a-6aa0750e5690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:56.631782 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c8feb-9ac0-4c03-815a-6aa0750e5690 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:56.632057 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c8feb-9ac0-4c03-815a-6aa0750e5690 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:56.632322 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c8feb-9ac0-4c03-815a-6aa0750e5690 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:56.682395 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:56.697985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c8feb-9ac0-4c03-815a-6aa0750e5690 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:56.699479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c8feb-9ac0-4c03-815a-6aa0750e5690 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:56.805547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 557/1128] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:27:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:27:56.812136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb2eed-4db7-4194-8b24-ae0e5b47a521 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:56.949872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb2eed-4db7-4194-8b24-ae0e5b47a521 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:56.974398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:56.985680 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fce1e-025a-4e43-adf3-b25dab3d22b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:56.986358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fce1e-025a-4e43-adf3-b25dab3d22b8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:56.986608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fce1e-025a-4e43-adf3-b25dab3d22b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:56.986873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fce1e-025a-4e43-adf3-b25dab3d22b8 None admin] PATH_INFO: `/v3/projects/0f479859e90d4428bc41bc4abbeca8db` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:56.988475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9fce1e-025a-4e43-adf3-b25dab3d22b8 None admin] RBAC: Authorizing `identity:get_project(project_id=0f479859e90d4428bc41bc4abbeca8db)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:56.989924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9fce1e-025a-4e43-adf3-b25dab3d22b8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:56.997860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 572/1129] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:27:56 2022] GET /identity/v3/projects/0f479859e90d4428bc41bc4abbeca8db => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:56.998279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c8feb-9ac0-4c03-815a-6aa0750e5690 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:57.083859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:57.100821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c8feb-9ac0-4c03-815a-6aa0750e5690 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:57.139856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c2c35-316f-4839-8390-726aaa073b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:57.140458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2c35-316f-4839-8390-726aaa073b14 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:57.140733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2c35-316f-4839-8390-726aaa073b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:57.140999 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2c35-316f-4839-8390-726aaa073b14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:57.141724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c2c35-316f-4839-8390-726aaa073b14 None admin] RBAC: Adding query filter params (role.id=65f33fe3045c433b9a3e045e7572aed6, scope.project.id=0f479859e90d4428bc41bc4abbeca8db, user.id=c472a30b303b421f9ffa689bdb4e3232) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:27:57.142028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c2c35-316f-4839-8390-726aaa073b14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:57.143582 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c2c35-316f-4839-8390-726aaa073b14 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:57.149974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 558/1130] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:27:56 2022] GET /identity/v3/role_assignments?user.id=c472a30b303b421f9ffa689bdb4e3232&scope.project.id=0f479859e90d4428bc41bc4abbeca8db&role.id=65f33fe3045c433b9a3e045e7572aed6 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:59.332677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d463f-16c0-4cd6-91bb-6187f90220e1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:59.333230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d463f-16c0-4cd6-91bb-6187f90220e1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:59.333230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d463f-16c0-4cd6-91bb-6187f90220e1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:59.334551 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 573/1131] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:27:59.339863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaac7e-58f0-4795-b4fc-11fd0d48fd40 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:59.340322 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaac7e-58f0-4795-b4fc-11fd0d48fd40 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:59.340424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcaac7e-58f0-4795-b4fc-11fd0d48fd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:59.359582 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dbcaac7e-58f0-4795-b4fc-11fd0d48fd40 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:27:59.360769 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dbcaac7e-58f0-4795-b4fc-11fd0d48fd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:27:59.425443 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 559/1132] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:27:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:27:59.690786 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d463f-16c0-4cd6-91bb-6187f90220e1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:59.789725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d463f-16c0-4cd6-91bb-6187f90220e1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:59.824678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785418c-2b78-460c-baa9-8c6a46f4d1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:59.825289 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785418c-2b78-460c-baa9-8c6a46f4d1df None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:59.825561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785418c-2b78-460c-baa9-8c6a46f4d1df None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:59.825823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785418c-2b78-460c-baa9-8c6a46f4d1df None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:59.826965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785418c-2b78-460c-baa9-8c6a46f4d1df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:59.828387 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785418c-2b78-460c-baa9-8c6a46f4d1df None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:27:59.829934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 574/1133] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:27:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:27:59.835788 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaac7e-58f0-4795-b4fc-11fd0d48fd40 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:27:59.912650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:27:59.933806 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcaac7e-58f0-4795-b4fc-11fd0d48fd40 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:27:59.967841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3160fdc-7789-4443-a118-aac4e0bea4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:27:59.968369 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3160fdc-7789-4443-a118-aac4e0bea4f3 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:27:59.968641 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3160fdc-7789-4443-a118-aac4e0bea4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:27:59.968905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3160fdc-7789-4443-a118-aac4e0bea4f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:27:59.969550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3160fdc-7789-4443-a118-aac4e0bea4f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:27:59.970984 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3160fdc-7789-4443-a118-aac4e0bea4f3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:00.063442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 560/1134] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:27:59 2022] POST /identity/v3/projects => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:28:02.236663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72991fba-deaf-425e-884a-d5f960cc8d21 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:02.237187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72991fba-deaf-425e-884a-d5f960cc8d21 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:02.237187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72991fba-deaf-425e-884a-d5f960cc8d21 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:02.238505 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 575/1135] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:28:02.243631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a2667-e373-4d40-b4cc-5c79dd13e3e4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:02.244092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a2667-e373-4d40-b4cc-5c79dd13e3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:02.244198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a2667-e373-4d40-b4cc-5c79dd13e3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:02.261877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-df2a2667-e373-4d40-b4cc-5c79dd13e3e4 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:28:02.263096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-df2a2667-e373-4d40-b4cc-5c79dd13e3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:28:02.328734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 561/1136] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:28:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:28:02.600419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72991fba-deaf-425e-884a-d5f960cc8d21 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:02.667458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:02.709874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72991fba-deaf-425e-884a-d5f960cc8d21 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:02.744509 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45eed3-53e4-4061-be94-dbc74dfde2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:02.745132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45eed3-53e4-4061-be94-dbc74dfde2eb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:02.745411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45eed3-53e4-4061-be94-dbc74dfde2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:02.745702 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45eed3-53e4-4061-be94-dbc74dfde2eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:02.746803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45eed3-53e4-4061-be94-dbc74dfde2eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:02.748736 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45eed3-53e4-4061-be94-dbc74dfde2eb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:02.750327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 576/1137] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:28:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:02.756096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a2667-e373-4d40-b4cc-5c79dd13e3e4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:02.813245 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:02.853331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a2667-e373-4d40-b4cc-5c79dd13e3e4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:02.887218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7721dce-e8c0-4d6c-97ae-e124de2bea62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:02.887816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7721dce-e8c0-4d6c-97ae-e124de2bea62 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:02.888087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7721dce-e8c0-4d6c-97ae-e124de2bea62 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:02.888353 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7721dce-e8c0-4d6c-97ae-e124de2bea62 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:02.889557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7721dce-e8c0-4d6c-97ae-e124de2bea62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:02.890990 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7721dce-e8c0-4d6c-97ae-e124de2bea62 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:02.956771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 562/1138] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:28:02 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:28:05.165894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b929b4-59e4-4f2b-83a6-224d45245d3b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:05.166445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b929b4-59e4-4f2b-83a6-224d45245d3b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:05.166445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b929b4-59e4-4f2b-83a6-224d45245d3b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:05.167692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 577/1139] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:28:05.173484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5322c8-30c3-4422-b225-f893f81bb171 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:05.173877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5322c8-30c3-4422-b225-f893f81bb171 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:05.174042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5322c8-30c3-4422-b225-f893f81bb171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:05.191855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8c5322c8-30c3-4422-b225-f893f81bb171 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:28:05.193046 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8c5322c8-30c3-4422-b225-f893f81bb171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:28:05.269169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 563/1140] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:28:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:28:05.541733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b929b4-59e4-4f2b-83a6-224d45245d3b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:05.712492 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b929b4-59e4-4f2b-83a6-224d45245d3b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:05.738033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:05.745424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145dda1-d196-4359-a95a-57f3d588a198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:05.751011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145dda1-d196-4359-a95a-57f3d588a198 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:05.751011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145dda1-d196-4359-a95a-57f3d588a198 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:05.751011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145dda1-d196-4359-a95a-57f3d588a198 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:05.756713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145dda1-d196-4359-a95a-57f3d588a198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f145dda1-d196-4359-a95a-57f3d588a198 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:28:05.761521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:28:05.762750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:28:05.764414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:28:05.764414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:28:05.764414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:28:05.764414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:28:05.764414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:28:05.764414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:28:05.764414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:28:05.764414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:28:05.764414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:28:05.764414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:28:05.764414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:28:05.764414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:28:05.764414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 578/1141] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:28:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:28:05.769345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5322c8-30c3-4422-b225-f893f81bb171 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:05.813492 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:05.863702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5322c8-30c3-4422-b225-f893f81bb171 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:05.897862 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2332c9-ef71-45dd-ba14-03e8516d880b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:05.898487 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2332c9-ef71-45dd-ba14-03e8516d880b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:05.898760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2332c9-ef71-45dd-ba14-03e8516d880b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:05.899023 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2332c9-ef71-45dd-ba14-03e8516d880b None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:05.899672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2332c9-ef71-45dd-ba14-03e8516d880b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:28:05.899965 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2332c9-ef71-45dd-ba14-03e8516d880b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:05.901305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2332c9-ef71-45dd-ba14-03e8516d880b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:05.909349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 564/1142] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:28:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:05.915277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145dda1-d196-4359-a95a-57f3d588a198 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:06.014841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145dda1-d196-4359-a95a-57f3d588a198 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:06.048379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36326f-4d9f-4985-a117-40b4dd6d5dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:06.048980 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36326f-4d9f-4985-a117-40b4dd6d5dfc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:06.049253 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36326f-4d9f-4985-a117-40b4dd6d5dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:06.049519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36326f-4d9f-4985-a117-40b4dd6d5dfc None admin] PATH_INFO: `/v3/users/5d86bb57720b4ed6b008af3de8110b91` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:06.062908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36326f-4d9f-4985-a117-40b4dd6d5dfc None admin] RBAC: Authorizing `identity:get_user(user_id=5d86bb57720b4ed6b008af3de8110b91)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:06.064388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36326f-4d9f-4985-a117-40b4dd6d5dfc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:06.065916 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 579/1143] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:28:05 2022] GET /identity/v3/users/5d86bb57720b4ed6b008af3de8110b91 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:06.071879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2332c9-ef71-45dd-ba14-03e8516d880b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:06.167581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2332c9-ef71-45dd-ba14-03e8516d880b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:06.178457 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:06.205389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7977e-f7c5-4ccc-81a9-6b7fca5333a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:06.205990 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7977e-f7c5-4ccc-81a9-6b7fca5333a4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:06.206289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7977e-f7c5-4ccc-81a9-6b7fca5333a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:06.206553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7977e-f7c5-4ccc-81a9-6b7fca5333a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:06.271581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7977e-f7c5-4ccc-81a9-6b7fca5333a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:06.273025 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7977e-f7c5-4ccc-81a9-6b7fca5333a4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:06.377808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 565/1144] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:28:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:28:06.384137 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36326f-4d9f-4985-a117-40b4dd6d5dfc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:06.430118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:06.480664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36326f-4d9f-4985-a117-40b4dd6d5dfc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:06.513193 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941f0b3-9582-43d2-8e4c-b6390468fee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:06.513825 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f0b3-9582-43d2-8e4c-b6390468fee2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:06.514123 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f0b3-9582-43d2-8e4c-b6390468fee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:06.514419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f0b3-9582-43d2-8e4c-b6390468fee2 None admin] PATH_INFO: `/v3/projects/3f966ef4400948248ede119529273248` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:06.515502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941f0b3-9582-43d2-8e4c-b6390468fee2 None admin] RBAC: Authorizing `identity:get_project(project_id=3f966ef4400948248ede119529273248)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:06.516915 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941f0b3-9582-43d2-8e4c-b6390468fee2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:06.518552 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 580/1145] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:28:06 2022] GET /identity/v3/projects/3f966ef4400948248ede119529273248 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:06.524821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7977e-f7c5-4ccc-81a9-6b7fca5333a4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:06.535339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:06.621202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7977e-f7c5-4ccc-81a9-6b7fca5333a4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:06.653491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1253f-6e95-46bc-905a-1f834deabd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:06.654027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1253f-6e95-46bc-905a-1f834deabd48 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:06.654845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1253f-6e95-46bc-905a-1f834deabd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:06.655191 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1253f-6e95-46bc-905a-1f834deabd48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:06.655978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1253f-6e95-46bc-905a-1f834deabd48 None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=3f966ef4400948248ede119529273248, user.id=5d86bb57720b4ed6b008af3de8110b91) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:28:06.656458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1253f-6e95-46bc-905a-1f834deabd48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:06.658754 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1253f-6e95-46bc-905a-1f834deabd48 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:06.667056 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 566/1146] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:28:06 2022] GET /identity/v3/role_assignments?user.id=5d86bb57720b4ed6b008af3de8110b91&scope.project.id=3f966ef4400948248ede119529273248&role.id=020bc02dca71485bab2312442adaa179 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:08.796127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a7931-9a33-461d-98d4-3f545b6175e3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:08.796693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a7931-9a33-461d-98d4-3f545b6175e3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:08.796693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a7931-9a33-461d-98d4-3f545b6175e3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:08.797966 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 581/1147] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:28:08.803061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396b491-6e45-41d7-8307-4f0483152b64 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:08.803454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396b491-6e45-41d7-8307-4f0483152b64 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:08.803607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396b491-6e45-41d7-8307-4f0483152b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:08.822000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8396b491-6e45-41d7-8307-4f0483152b64 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:28:08.823229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8396b491-6e45-41d7-8307-4f0483152b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:28:08.888295 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 567/1148] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:28:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:28:09.158263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a7931-9a33-461d-98d4-3f545b6175e3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:09.257297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a7931-9a33-461d-98d4-3f545b6175e3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:09.277369 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:09.299921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd9a89-f175-48f8-9f47-84ee2ba622eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:09.301642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd9a89-f175-48f8-9f47-84ee2ba622eb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:09.301642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd9a89-f175-48f8-9f47-84ee2ba622eb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:09.301642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd9a89-f175-48f8-9f47-84ee2ba622eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:09.312350 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd9a89-f175-48f8-9f47-84ee2ba622eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5ecd9a89-f175-48f8-9f47-84ee2ba622eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:28:09.318301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:28:09.319392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:28:09.320504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:28:09.320504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:28:09.320504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:28:09.320504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:28:09.320504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:28:09.320504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:28:09.320504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:28:09.320504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:28:09.320504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:28:09.320504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:28:09.320504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:28:09.320504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:28:09.320504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 582/1149] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:28:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:28:09.326968 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396b491-6e45-41d7-8307-4f0483152b64 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:09.329138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:09.422981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396b491-6e45-41d7-8307-4f0483152b64 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:09.455709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4668d-7685-4920-9fce-54e2129ac482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:09.456309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4668d-7685-4920-9fce-54e2129ac482 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:09.456584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4668d-7685-4920-9fce-54e2129ac482 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:09.456850 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4668d-7685-4920-9fce-54e2129ac482 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:09.457504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4668d-7685-4920-9fce-54e2129ac482 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:28:09.457801 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4668d-7685-4920-9fce-54e2129ac482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:09.459177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4668d-7685-4920-9fce-54e2129ac482 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:09.467656 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 568/1150] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:28:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:09.473657 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd9a89-f175-48f8-9f47-84ee2ba622eb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:09.580332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd9a89-f175-48f8-9f47-84ee2ba622eb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:09.626681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e39acf-38cd-450a-813f-8640b826e184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:09.627316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e39acf-38cd-450a-813f-8640b826e184 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:09.627592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e39acf-38cd-450a-813f-8640b826e184 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:09.627860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e39acf-38cd-450a-813f-8640b826e184 None admin] PATH_INFO: `/v3/users/5d86bb57720b4ed6b008af3de8110b91` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:09.628992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e39acf-38cd-450a-813f-8640b826e184 None admin] RBAC: Authorizing `identity:get_user(user_id=5d86bb57720b4ed6b008af3de8110b91)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:09.630579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e39acf-38cd-450a-813f-8640b826e184 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:09.632172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 583/1151] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:28:09 2022] GET /identity/v3/users/5d86bb57720b4ed6b008af3de8110b91 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:09.638123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4668d-7685-4920-9fce-54e2129ac482 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:09.700418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:09.737793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4668d-7685-4920-9fce-54e2129ac482 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:09.769921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd6fae-c4a4-4ac9-84d3-d60edeaf3966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:09.770564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd6fae-c4a4-4ac9-84d3-d60edeaf3966 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:09.770841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd6fae-c4a4-4ac9-84d3-d60edeaf3966 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:09.771105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd6fae-c4a4-4ac9-84d3-d60edeaf3966 None admin] PATH_INFO: `/v3/projects/3f966ef4400948248ede119529273248` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:09.772199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd6fae-c4a4-4ac9-84d3-d60edeaf3966 None admin] RBAC: Authorizing `identity:get_project(project_id=3f966ef4400948248ede119529273248)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:09.773620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd6fae-c4a4-4ac9-84d3-d60edeaf3966 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:09.775279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 569/1152] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:28:09 2022] GET /identity/v3/projects/3f966ef4400948248ede119529273248 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:09.781889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e39acf-38cd-450a-813f-8640b826e184 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:09.823741 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:09.879515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e39acf-38cd-450a-813f-8640b826e184 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:09.913488 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be89176-8efb-408f-8b03-e156174a0679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:09.914134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be89176-8efb-408f-8b03-e156174a0679 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:09.914452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be89176-8efb-408f-8b03-e156174a0679 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:09.914727 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be89176-8efb-408f-8b03-e156174a0679 None admin] PATH_INFO: `/v3/projects/3f966ef4400948248ede119529273248/users/5d86bb57720b4ed6b008af3de8110b91/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:09.916598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be89176-8efb-408f-8b03-e156174a0679 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f966ef4400948248ede119529273248, user_id=5d86bb57720b4ed6b008af3de8110b91, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:09.918019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be89176-8efb-408f-8b03-e156174a0679 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:10.018303 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 584/1153] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:28:09 2022] PUT /identity/v3/projects/3f966ef4400948248ede119529273248/users/5d86bb57720b4ed6b008af3de8110b91/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:28:12.190302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538f7c3-30d2-4b9d-9ca9-75904e59ddfa None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:12.190815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538f7c3-30d2-4b9d-9ca9-75904e59ddfa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:12.190815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538f7c3-30d2-4b9d-9ca9-75904e59ddfa None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:12.192096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 570/1154] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:28:12.197391 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faea578-418d-4f7f-bebd-4442955a3971 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:12.197788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faea578-418d-4f7f-bebd-4442955a3971 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:12.197927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faea578-418d-4f7f-bebd-4442955a3971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:12.217381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8faea578-418d-4f7f-bebd-4442955a3971 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:28:12.218594 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8faea578-418d-4f7f-bebd-4442955a3971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:28:12.284557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 585/1155] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:28:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:28:12.551782 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538f7c3-30d2-4b9d-9ca9-75904e59ddfa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:12.652323 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538f7c3-30d2-4b9d-9ca9-75904e59ddfa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:12.679701 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:12.684503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4555a5-3d9c-4a05-92a9-fb84c146a784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:12.685131 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4555a5-3d9c-4a05-92a9-fb84c146a784 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:12.685922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4555a5-3d9c-4a05-92a9-fb84c146a784 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:12.686235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4555a5-3d9c-4a05-92a9-fb84c146a784 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:12.696803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4555a5-3d9c-4a05-92a9-fb84c146a784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4b4555a5-3d9c-4a05-92a9-fb84c146a784 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:28:12.702328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:28:12.703422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:28:12.704529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:28:12.704529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:28:12.704529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:28:12.704529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:28:12.704529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:28:12.704529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:28:12.704529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:28:12.704529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:28:12.704529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:28:12.704529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:28:12.704529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:28:12.704529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:28:12.704529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 571/1156] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:28:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:28:12.709893 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faea578-418d-4f7f-bebd-4442955a3971 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:12.740265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:12.808686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faea578-418d-4f7f-bebd-4442955a3971 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:12.842431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3ce98-b22e-484b-a924-31acd692f8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:12.843025 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3ce98-b22e-484b-a924-31acd692f8fc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:12.843296 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3ce98-b22e-484b-a924-31acd692f8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:12.843560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3ce98-b22e-484b-a924-31acd692f8fc None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:12.844212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3ce98-b22e-484b-a924-31acd692f8fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:28:12.844507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3ce98-b22e-484b-a924-31acd692f8fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:12.845854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3ce98-b22e-484b-a924-31acd692f8fc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:12.853642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 586/1157] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:28:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:12.860423 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4555a5-3d9c-4a05-92a9-fb84c146a784 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:12.957280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4555a5-3d9c-4a05-92a9-fb84c146a784 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:12.989472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db9c24-77e2-4f34-9bbd-426d96b7cc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:12.990084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db9c24-77e2-4f34-9bbd-426d96b7cc58 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:12.990385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db9c24-77e2-4f34-9bbd-426d96b7cc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:12.990650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db9c24-77e2-4f34-9bbd-426d96b7cc58 None admin] PATH_INFO: `/v3/users/5d86bb57720b4ed6b008af3de8110b91` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:12.991733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db9c24-77e2-4f34-9bbd-426d96b7cc58 None admin] RBAC: Authorizing `identity:get_user(user_id=5d86bb57720b4ed6b008af3de8110b91)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:12.993250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db9c24-77e2-4f34-9bbd-426d96b7cc58 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:12.994838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 572/1158] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:28:12 2022] GET /identity/v3/users/5d86bb57720b4ed6b008af3de8110b91 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:13.001188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3ce98-b22e-484b-a924-31acd692f8fc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:13.096076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:13.102978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3ce98-b22e-484b-a924-31acd692f8fc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:13.137838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de9a74-7f14-4fee-9623-75847fec9110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:13.138517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9a74-7f14-4fee-9623-75847fec9110 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:13.138810 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9a74-7f14-4fee-9623-75847fec9110 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:13.139083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de9a74-7f14-4fee-9623-75847fec9110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:13.206172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de9a74-7f14-4fee-9623-75847fec9110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:13.207646 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de9a74-7f14-4fee-9623-75847fec9110 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:13.299625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:13.307091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 587/1159] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:28:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:28:13.313306 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db9c24-77e2-4f34-9bbd-426d96b7cc58 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:13.374531 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:13.413949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db9c24-77e2-4f34-9bbd-426d96b7cc58 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:13.446213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca5707-7f8c-47f2-810a-894684f4826e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:13.446858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5707-7f8c-47f2-810a-894684f4826e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:13.447129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5707-7f8c-47f2-810a-894684f4826e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:13.447394 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5707-7f8c-47f2-810a-894684f4826e None admin] PATH_INFO: `/v3/projects/3f966ef4400948248ede119529273248` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:13.448485 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca5707-7f8c-47f2-810a-894684f4826e None admin] RBAC: Authorizing `identity:get_project(project_id=3f966ef4400948248ede119529273248)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:13.449890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca5707-7f8c-47f2-810a-894684f4826e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:13.452097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 573/1160] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:28:13 2022] GET /identity/v3/projects/3f966ef4400948248ede119529273248 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:13.458357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de9a74-7f14-4fee-9623-75847fec9110 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:13.554691 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de9a74-7f14-4fee-9623-75847fec9110 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:13.608670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacfe6e4-4ca1-4191-a7cd-8f6db7dbd462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:13.609704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfe6e4-4ca1-4191-a7cd-8f6db7dbd462 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:13.610238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfe6e4-4ca1-4191-a7cd-8f6db7dbd462 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:13.610644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfe6e4-4ca1-4191-a7cd-8f6db7dbd462 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:13.612017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacfe6e4-4ca1-4191-a7cd-8f6db7dbd462 None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=3f966ef4400948248ede119529273248, user.id=5d86bb57720b4ed6b008af3de8110b91) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:28:13.612572 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacfe6e4-4ca1-4191-a7cd-8f6db7dbd462 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:13.615384 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacfe6e4-4ca1-4191-a7cd-8f6db7dbd462 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:13.625414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 588/1161] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:28:13 2022] GET /identity/v3/role_assignments?user.id=5d86bb57720b4ed6b008af3de8110b91&scope.project.id=3f966ef4400948248ede119529273248&role.id=020bc02dca71485bab2312442adaa179 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:15.809071 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f561783-3cea-465e-b94d-a4098c79203d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:15.809630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f561783-3cea-465e-b94d-a4098c79203d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:15.809630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f561783-3cea-465e-b94d-a4098c79203d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:15.811597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 574/1162] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:28:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:28:15.816827 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974125ca-601c-4731-afe6-dda1d8845bc7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:15.817207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974125ca-601c-4731-afe6-dda1d8845bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:15.817381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974125ca-601c-4731-afe6-dda1d8845bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:15.835275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-974125ca-601c-4731-afe6-dda1d8845bc7 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:28:15.836460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-974125ca-601c-4731-afe6-dda1d8845bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:28:15.890493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:15.903735 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 589/1163] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:28:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:28:16.173554 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f561783-3cea-465e-b94d-a4098c79203d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:16.272317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f561783-3cea-465e-b94d-a4098c79203d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:16.301302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:16.309441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f93624-e7f6-4c34-a4a6-a3b571bd2df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:16.310062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f93624-e7f6-4c34-a4a6-a3b571bd2df0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:16.310393 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f93624-e7f6-4c34-a4a6-a3b571bd2df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:16.310666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f93624-e7f6-4c34-a4a6-a3b571bd2df0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:16.376437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f93624-e7f6-4c34-a4a6-a3b571bd2df0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:16.377859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f93624-e7f6-4c34-a4a6-a3b571bd2df0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:16.474384 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 575/1164] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:28:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:28:16.480301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974125ca-601c-4731-afe6-dda1d8845bc7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:16.574393 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974125ca-601c-4731-afe6-dda1d8845bc7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:16.607490 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd5d2e-5098-42e0-9225-30389e82efb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:16.608093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd5d2e-5098-42e0-9225-30389e82efb6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:16.608365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd5d2e-5098-42e0-9225-30389e82efb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:16.608628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd5d2e-5098-42e0-9225-30389e82efb6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:16.615074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd5d2e-5098-42e0-9225-30389e82efb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:16.616493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd5d2e-5098-42e0-9225-30389e82efb6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3bbd5d2e-5098-42e0-9225-30389e82efb6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:28:16.627476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:28:16.628692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:28:16.629766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 590/1165] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 08:28:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:28:16.634858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f93624-e7f6-4c34-a4a6-a3b571bd2df0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:16.662156 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:16.732214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f93624-e7f6-4c34-a4a6-a3b571bd2df0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:16.764669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb6632-0268-4745-a1f7-9edbd092b57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:16.765262 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb6632-0268-4745-a1f7-9edbd092b57e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:16.765537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb6632-0268-4745-a1f7-9edbd092b57e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:16.765805 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb6632-0268-4745-a1f7-9edbd092b57e None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:16.766473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb6632-0268-4745-a1f7-9edbd092b57e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:28:16.766774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb6632-0268-4745-a1f7-9edbd092b57e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:16.768136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb6632-0268-4745-a1f7-9edbd092b57e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:16.775592 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 576/1166] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 08:28:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:18.913962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4894a-9daa-4e86-a305-8cf467f92da5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:18.914481 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4894a-9daa-4e86-a305-8cf467f92da5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:18.914601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4894a-9daa-4e86-a305-8cf467f92da5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:18.915860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 591/1167] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:28:18.921717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cfc08-89d8-4479-bb73-a37ccea8566d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:18.922115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cfc08-89d8-4479-bb73-a37ccea8566d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:18.922288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cfc08-89d8-4479-bb73-a37ccea8566d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:18.940811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d96cfc08-89d8-4479-bb73-a37ccea8566d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:28:18.942003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d96cfc08-89d8-4479-bb73-a37ccea8566d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:28:19.007114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 577/1168] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:28:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:28:19.281143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4894a-9daa-4e86-a305-8cf467f92da5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:19.330241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:19.385704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4894a-9daa-4e86-a305-8cf467f92da5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:19.418886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff9042-f49e-41a8-857c-86f1c7513bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:19.419481 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff9042-f49e-41a8-857c-86f1c7513bf0 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:19.420306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff9042-f49e-41a8-857c-86f1c7513bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:19.420574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff9042-f49e-41a8-857c-86f1c7513bf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:19.421239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff9042-f49e-41a8-857c-86f1c7513bf0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:19.422643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff9042-f49e-41a8-857c-86f1c7513bf0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:19.476929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 592/1169] 10.222.0.44 () {60 vars in 1138 bytes} [Sun Sep 11 08:28:19 2022] POST /identity/v3/domains => generated 267 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:28:21.655371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3ac56-acbc-4bc7-8042-f9645384f448 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:21.655911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3ac56-acbc-4bc7-8042-f9645384f448 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:21.655911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3ac56-acbc-4bc7-8042-f9645384f448 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:21.657225 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 578/1170] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:28:21.662560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ea2f-889e-43e8-957d-53236a419e7c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:21.662970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ea2f-889e-43e8-957d-53236a419e7c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:21.663101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547ea2f-889e-43e8-957d-53236a419e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:21.681230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3547ea2f-889e-43e8-957d-53236a419e7c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:28:21.682465 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3547ea2f-889e-43e8-957d-53236a419e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:28:21.749755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 593/1171] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:28:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:28:22.029117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3ac56-acbc-4bc7-8042-f9645384f448 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:22.052820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:22.140027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3ac56-acbc-4bc7-8042-f9645384f448 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:22.173070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61249e16-ee0f-4708-9beb-28e0cb957c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:22.173815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61249e16-ee0f-4708-9beb-28e0cb957c3a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:22.174087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61249e16-ee0f-4708-9beb-28e0cb957c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:22.174377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61249e16-ee0f-4708-9beb-28e0cb957c3a None admin] PATH_INFO: `/v3/domains/c60c4f651bb348cb88d2bbc788a0d16d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:22.184274 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61249e16-ee0f-4708-9beb-28e0cb957c3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=c60c4f651bb348cb88d2bbc788a0d16d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:22.186324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61249e16-ee0f-4708-9beb-28e0cb957c3a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:22.187909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 579/1172] 10.222.0.44 () {56 vars in 1183 bytes} [Sun Sep 11 08:28:22 2022] GET /identity/v3/domains/c60c4f651bb348cb88d2bbc788a0d16d => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:22.193762 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547ea2f-889e-43e8-957d-53236a419e7c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:22.231519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:22.292717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547ea2f-889e-43e8-957d-53236a419e7c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:22.330949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4f3e6-64ad-496b-bbdb-7374a95168fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:22.331549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4f3e6-64ad-496b-bbdb-7374a95168fa None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:22.331822 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4f3e6-64ad-496b-bbdb-7374a95168fa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:22.332089 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4f3e6-64ad-496b-bbdb-7374a95168fa None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:22.332736 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4f3e6-64ad-496b-bbdb-7374a95168fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:22.334180 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4f3e6-64ad-496b-bbdb-7374a95168fa None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:22.415368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 594/1173] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:28:22 2022] POST /identity/v3/projects => generated 361 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:28:24.625411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439caafa-8c1d-4cb6-9636-a10e78a286b0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:24.625983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439caafa-8c1d-4cb6-9636-a10e78a286b0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:24.626079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439caafa-8c1d-4cb6-9636-a10e78a286b0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:24.627426 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 580/1174] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:28:24.632877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc4d23-ba96-4fba-8845-293300c57740 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:24.633312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc4d23-ba96-4fba-8845-293300c57740 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:24.633504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc4d23-ba96-4fba-8845-293300c57740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:24.651814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-febc4d23-ba96-4fba-8845-293300c57740 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:28:24.653009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-febc4d23-ba96-4fba-8845-293300c57740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:28:24.721874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 595/1175] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:28:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:28:24.990893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439caafa-8c1d-4cb6-9636-a10e78a286b0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:25.088326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:25.111562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439caafa-8c1d-4cb6-9636-a10e78a286b0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:25.150912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9af14c-0320-473b-b3c4-8f3643f9af8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:25.151539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9af14c-0320-473b-b3c4-8f3643f9af8a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:25.151819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9af14c-0320-473b-b3c4-8f3643f9af8a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:25.152092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9af14c-0320-473b-b3c4-8f3643f9af8a None admin] PATH_INFO: `/v3/domains/c60c4f651bb348cb88d2bbc788a0d16d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:25.153205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9af14c-0320-473b-b3c4-8f3643f9af8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=c60c4f651bb348cb88d2bbc788a0d16d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:25.155238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9af14c-0320-473b-b3c4-8f3643f9af8a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:25.156830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 581/1176] 10.222.0.44 () {56 vars in 1183 bytes} [Sun Sep 11 08:28:24 2022] GET /identity/v3/domains/c60c4f651bb348cb88d2bbc788a0d16d => generated 267 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:25.162686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc4d23-ba96-4fba-8845-293300c57740 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:25.183416 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:25.260959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc4d23-ba96-4fba-8845-293300c57740 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:25.295223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc73d77-c410-4215-9de1-a3a85ff1e445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:25.295810 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc73d77-c410-4215-9de1-a3a85ff1e445 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:25.296081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc73d77-c410-4215-9de1-a3a85ff1e445 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:25.296345 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc73d77-c410-4215-9de1-a3a85ff1e445 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:25.296971 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc73d77-c410-4215-9de1-a3a85ff1e445 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:25.298403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc73d77-c410-4215-9de1-a3a85ff1e445 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:25.370560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 596/1177] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:28:25 2022] POST /identity/v3/users => generated 311 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:28:27.555404 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166fc35-0077-4dfc-8c26-fa6559ed2f88 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:27.555940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166fc35-0077-4dfc-8c26-fa6559ed2f88 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:27.555940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166fc35-0077-4dfc-8c26-fa6559ed2f88 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:27.557292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 582/1178] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:28:27.563241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f72518-db9d-4cdf-a82c-27f0362e2bbb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:27.563668 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f72518-db9d-4cdf-a82c-27f0362e2bbb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:27.563783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f72518-db9d-4cdf-a82c-27f0362e2bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:27.581549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f4f72518-db9d-4cdf-a82c-27f0362e2bbb None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:28:27.582759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f4f72518-db9d-4cdf-a82c-27f0362e2bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:28:27.646637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 597/1179] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:28:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:28:27.920451 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166fc35-0077-4dfc-8c26-fa6559ed2f88 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:28.017387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166fc35-0077-4dfc-8c26-fa6559ed2f88 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:28.051215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cab536-9350-4284-b0bc-82d44aa01aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:28.051824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cab536-9350-4284-b0bc-82d44aa01aaf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:28.052094 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cab536-9350-4284-b0bc-82d44aa01aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:28.052358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cab536-9350-4284-b0bc-82d44aa01aaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:28.062051 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cab536-9350-4284-b0bc-82d44aa01aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-86cab536-9350-4284-b0bc-82d44aa01aaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:28:28.066891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:28:28.068061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:28:28.069166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:28:28.069166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:28:28.069166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:28:28.069166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:28:28.069166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:28:28.069166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:28:28.069166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:28:28.069166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:28:28.069166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:28:28.069166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:28:28.069166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:28:28.069166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:28:28.069166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 583/1180] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:28:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:28:28.074078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f72518-db9d-4cdf-a82c-27f0362e2bbb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:28.082578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:28.172383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f72518-db9d-4cdf-a82c-27f0362e2bbb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:28.205373 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b22625-8e3d-463e-a31d-e2fc89ea2688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:28.205976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b22625-8e3d-463e-a31d-e2fc89ea2688 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:28.206286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b22625-8e3d-463e-a31d-e2fc89ea2688 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:28.206557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b22625-8e3d-463e-a31d-e2fc89ea2688 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:28.207215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b22625-8e3d-463e-a31d-e2fc89ea2688 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:28:28.207514 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b22625-8e3d-463e-a31d-e2fc89ea2688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:28.208878 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b22625-8e3d-463e-a31d-e2fc89ea2688 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:28.216789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 598/1181] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:28:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:28.223150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cab536-9350-4284-b0bc-82d44aa01aaf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:28.236141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:28.321072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cab536-9350-4284-b0bc-82d44aa01aaf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:28.353385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553aec6d-0b4f-4315-a7f5-f551946bf145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:28.354004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553aec6d-0b4f-4315-a7f5-f551946bf145 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:28.354310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553aec6d-0b4f-4315-a7f5-f551946bf145 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:28.354578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553aec6d-0b4f-4315-a7f5-f551946bf145 None admin] PATH_INFO: `/v3/users/57cb7c95f1234265a121885cd1cbfa0d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:28.367908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553aec6d-0b4f-4315-a7f5-f551946bf145 None admin] RBAC: Authorizing `identity:get_user(user_id=57cb7c95f1234265a121885cd1cbfa0d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:28.369456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553aec6d-0b4f-4315-a7f5-f551946bf145 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:28.371106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 584/1182] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:28:28 2022] GET /identity/v3/users/57cb7c95f1234265a121885cd1cbfa0d => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:28.377581 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b22625-8e3d-463e-a31d-e2fc89ea2688 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:28.441990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:28.475261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b22625-8e3d-463e-a31d-e2fc89ea2688 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:28.507911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c6ad8-f0e5-4502-99a4-d5a890725de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:28.508499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c6ad8-f0e5-4502-99a4-d5a890725de0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:28.508769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c6ad8-f0e5-4502-99a4-d5a890725de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:28.509032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c6ad8-f0e5-4502-99a4-d5a890725de0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:28.573180 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c6ad8-f0e5-4502-99a4-d5a890725de0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:28.574645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c6ad8-f0e5-4502-99a4-d5a890725de0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:28.640657 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:28.673761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 599/1183] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:28:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:28:28.679836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553aec6d-0b4f-4315-a7f5-f551946bf145 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:28.747814 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:28.780820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553aec6d-0b4f-4315-a7f5-f551946bf145 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:28.814665 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2748baa-849f-4c2a-abd6-7aec58a3e279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:28.815299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2748baa-849f-4c2a-abd6-7aec58a3e279 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:28.815569 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2748baa-849f-4c2a-abd6-7aec58a3e279 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:28.815830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2748baa-849f-4c2a-abd6-7aec58a3e279 None admin] PATH_INFO: `/v3/projects/0cc6e70988094ce486a8b423cc6e1d04` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:28.816969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2748baa-849f-4c2a-abd6-7aec58a3e279 None admin] RBAC: Authorizing `identity:get_project(project_id=0cc6e70988094ce486a8b423cc6e1d04)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:28.818405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2748baa-849f-4c2a-abd6-7aec58a3e279 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:28.820253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 585/1184] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:28:28 2022] GET /identity/v3/projects/0cc6e70988094ce486a8b423cc6e1d04 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:28.827151 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c6ad8-f0e5-4502-99a4-d5a890725de0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:28.925249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c6ad8-f0e5-4502-99a4-d5a890725de0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:28.958709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba4cb7-a60d-4237-8545-7483e8a6051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:28.959297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba4cb7-a60d-4237-8545-7483e8a6051c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:28.959569 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba4cb7-a60d-4237-8545-7483e8a6051c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:28.959834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba4cb7-a60d-4237-8545-7483e8a6051c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:28.960537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba4cb7-a60d-4237-8545-7483e8a6051c None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=0cc6e70988094ce486a8b423cc6e1d04, user.id=57cb7c95f1234265a121885cd1cbfa0d) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:28:28.960838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba4cb7-a60d-4237-8545-7483e8a6051c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:28.962283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba4cb7-a60d-4237-8545-7483e8a6051c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:28.968309 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 600/1185] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:28:28 2022] GET /identity/v3/role_assignments?user.id=57cb7c95f1234265a121885cd1cbfa0d&scope.project.id=0cc6e70988094ce486a8b423cc6e1d04&role.id=020bc02dca71485bab2312442adaa179 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:28:31.116340 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34c791-0d70-4d13-ae32-655fec684c92 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:31.116907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34c791-0d70-4d13-ae32-655fec684c92 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:31.116907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34c791-0d70-4d13-ae32-655fec684c92 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:31.118238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 586/1186] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:28:31.124083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd12a5b-3782-48ca-bfb4-9860beba523a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:31.124511 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd12a5b-3782-48ca-bfb4-9860beba523a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:31.124612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd12a5b-3782-48ca-bfb4-9860beba523a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:31.142481 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8cd12a5b-3782-48ca-bfb4-9860beba523a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:28:31.143671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8cd12a5b-3782-48ca-bfb4-9860beba523a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:28:31.168409 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:31.208768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 601/1187] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:28:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:28:31.473391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34c791-0d70-4d13-ae32-655fec684c92 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:31.571659 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34c791-0d70-4d13-ae32-655fec684c92 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:31.605298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd00dad-db1a-47ea-8d92-8b5d151e3c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:31.605917 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd00dad-db1a-47ea-8d92-8b5d151e3c15 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:31.606230 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd00dad-db1a-47ea-8d92-8b5d151e3c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:31.606504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd00dad-db1a-47ea-8d92-8b5d151e3c15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:31.609700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:31.617427 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd00dad-db1a-47ea-8d92-8b5d151e3c15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8cd00dad-db1a-47ea-8d92-8b5d151e3c15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:28:31.626281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:28:31.627621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:28:31.628623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:28:31.628623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:28:31.628623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:28:31.628623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:28:31.628623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:28:31.628623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:28:31.628623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:28:31.628623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:28:31.628623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:28:31.628623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:28:31.628623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:28:31.628623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:28:31.628623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 587/1188] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:28:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:28:31.629649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd12a5b-3782-48ca-bfb4-9860beba523a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:31.726712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd12a5b-3782-48ca-bfb4-9860beba523a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:31.760017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564bb2d-3996-4456-a2bf-90ff37a3f7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:31.760614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564bb2d-3996-4456-a2bf-90ff37a3f7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:31.760886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564bb2d-3996-4456-a2bf-90ff37a3f7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:31.761152 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564bb2d-3996-4456-a2bf-90ff37a3f7a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:31.761800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564bb2d-3996-4456-a2bf-90ff37a3f7a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:28:31.762095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564bb2d-3996-4456-a2bf-90ff37a3f7a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:31.763515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564bb2d-3996-4456-a2bf-90ff37a3f7a8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:31.771521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 602/1189] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:28:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:31.777452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd00dad-db1a-47ea-8d92-8b5d151e3c15 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:31.874170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd00dad-db1a-47ea-8d92-8b5d151e3c15 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:31.907611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87291b-e875-4a07-a342-96a6da1b4ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:31.908227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87291b-e875-4a07-a342-96a6da1b4ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:31.908501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87291b-e875-4a07-a342-96a6da1b4ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:31.908772 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87291b-e875-4a07-a342-96a6da1b4ca8 None admin] PATH_INFO: `/v3/users/57cb7c95f1234265a121885cd1cbfa0d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:31.909867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87291b-e875-4a07-a342-96a6da1b4ca8 None admin] RBAC: Authorizing `identity:get_user(user_id=57cb7c95f1234265a121885cd1cbfa0d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:31.911308 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87291b-e875-4a07-a342-96a6da1b4ca8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:31.912889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 588/1190] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:28:31 2022] GET /identity/v3/users/57cb7c95f1234265a121885cd1cbfa0d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:31.918918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564bb2d-3996-4456-a2bf-90ff37a3f7a8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:31.944102 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:32.021466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564bb2d-3996-4456-a2bf-90ff37a3f7a8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:32.054494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127c42b-0f21-42f6-80e3-52493f5fe6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:32.055117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127c42b-0f21-42f6-80e3-52493f5fe6e1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:32.055395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127c42b-0f21-42f6-80e3-52493f5fe6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:32.055668 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127c42b-0f21-42f6-80e3-52493f5fe6e1 None admin] PATH_INFO: `/v3/projects/0cc6e70988094ce486a8b423cc6e1d04` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:32.056763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127c42b-0f21-42f6-80e3-52493f5fe6e1 None admin] RBAC: Authorizing `identity:get_project(project_id=0cc6e70988094ce486a8b423cc6e1d04)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:32.058220 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127c42b-0f21-42f6-80e3-52493f5fe6e1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:32.059850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 603/1191] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:28:31 2022] GET /identity/v3/projects/0cc6e70988094ce486a8b423cc6e1d04 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:32.065816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87291b-e875-4a07-a342-96a6da1b4ca8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:32.126908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:32.166808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87291b-e875-4a07-a342-96a6da1b4ca8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:32.199961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce76cd-d572-4c01-bd28-85069ed3b5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:32.201143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce76cd-d572-4c01-bd28-85069ed3b5ea None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:32.201413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce76cd-d572-4c01-bd28-85069ed3b5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:32.201677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce76cd-d572-4c01-bd28-85069ed3b5ea None admin] PATH_INFO: `/v3/projects/0cc6e70988094ce486a8b423cc6e1d04/users/57cb7c95f1234265a121885cd1cbfa0d/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:32.203560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce76cd-d572-4c01-bd28-85069ed3b5ea None admin] RBAC: Authorizing `identity:create_grant(project_id=0cc6e70988094ce486a8b423cc6e1d04, user_id=57cb7c95f1234265a121885cd1cbfa0d, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:32.204969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce76cd-d572-4c01-bd28-85069ed3b5ea None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:32.296456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 589/1192] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:28:32 2022] PUT /identity/v3/projects/0cc6e70988094ce486a8b423cc6e1d04/users/57cb7c95f1234265a121885cd1cbfa0d/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:28:34.425788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda2d7f-c231-4296-a20a-b9f51ae0def8 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:34.426373 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda2d7f-c231-4296-a20a-b9f51ae0def8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:34.426373 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda2d7f-c231-4296-a20a-b9f51ae0def8 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:34.427721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 604/1193] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:28:34.432975 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1047ff-d055-484d-b3a1-7ba457b016e4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:34.433380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1047ff-d055-484d-b3a1-7ba457b016e4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:34.433509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1047ff-d055-484d-b3a1-7ba457b016e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:34.451995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ab1047ff-d055-484d-b3a1-7ba457b016e4 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:28:34.453172 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ab1047ff-d055-484d-b3a1-7ba457b016e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:28:34.517853 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 590/1194] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:28:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:28:34.783845 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda2d7f-c231-4296-a20a-b9f51ae0def8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:34.880363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:34.890288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda2d7f-c231-4296-a20a-b9f51ae0def8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:34.923992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e9032-01a4-4e7d-ab7d-adb28d2fd315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:34.924611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e9032-01a4-4e7d-ab7d-adb28d2fd315 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:34.924887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e9032-01a4-4e7d-ab7d-adb28d2fd315 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:34.925157 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e9032-01a4-4e7d-ab7d-adb28d2fd315 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:34.934897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e9032-01a4-4e7d-ab7d-adb28d2fd315 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-256e9032-01a4-4e7d-ab7d-adb28d2fd315 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:28:34.939697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:28:34.940859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:28:34.942084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:28:34.942084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:28:34.942084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:28:34.942084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:28:34.942084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:28:34.942084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:28:34.942084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:28:34.942084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:28:34.942084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:28:34.942084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:28:34.942084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:28:34.942084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:28:34.942084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 605/1195] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:28:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:28:34.946986 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1047ff-d055-484d-b3a1-7ba457b016e4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:35.001446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:35.045534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1047ff-d055-484d-b3a1-7ba457b016e4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:35.079364 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffcc97-8732-441e-a6b7-84d24a4c6bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:35.079967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffcc97-8732-441e-a6b7-84d24a4c6bdf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:35.080246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffcc97-8732-441e-a6b7-84d24a4c6bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:35.080516 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffcc97-8732-441e-a6b7-84d24a4c6bdf None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:35.081173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffcc97-8732-441e-a6b7-84d24a4c6bdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:28:35.081475 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffcc97-8732-441e-a6b7-84d24a4c6bdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:35.082858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffcc97-8732-441e-a6b7-84d24a4c6bdf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:35.090688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 591/1196] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:28:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:35.097587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e9032-01a4-4e7d-ab7d-adb28d2fd315 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:35.195449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e9032-01a4-4e7d-ab7d-adb28d2fd315 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:35.229533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33549228-1f0c-4f46-9267-e2845d82f9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:35.230181 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33549228-1f0c-4f46-9267-e2845d82f9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:35.230458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33549228-1f0c-4f46-9267-e2845d82f9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:35.230724 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33549228-1f0c-4f46-9267-e2845d82f9f9 None admin] PATH_INFO: `/v3/users/57cb7c95f1234265a121885cd1cbfa0d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:35.231820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33549228-1f0c-4f46-9267-e2845d82f9f9 None admin] RBAC: Authorizing `identity:get_user(user_id=57cb7c95f1234265a121885cd1cbfa0d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:35.233280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33549228-1f0c-4f46-9267-e2845d82f9f9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:35.234889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 606/1197] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 08:28:35 2022] GET /identity/v3/users/57cb7c95f1234265a121885cd1cbfa0d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:35.240746 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffcc97-8732-441e-a6b7-84d24a4c6bdf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:35.338517 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffcc97-8732-441e-a6b7-84d24a4c6bdf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:35.354011 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:35.370658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eb0d8-04c1-4772-9931-c86e4c92d457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:35.371261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eb0d8-04c1-4772-9931-c86e4c92d457 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:35.371533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eb0d8-04c1-4772-9931-c86e4c92d457 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:35.371796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eb0d8-04c1-4772-9931-c86e4c92d457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:35.437169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861eb0d8-04c1-4772-9931-c86e4c92d457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:35.438633 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861eb0d8-04c1-4772-9931-c86e4c92d457 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:35.543927 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 592/1198] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:28:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:28:35.549917 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33549228-1f0c-4f46-9267-e2845d82f9f9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:35.572454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:35.649492 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33549228-1f0c-4f46-9267-e2845d82f9f9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:35.681911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b5f55-8faf-4983-b160-57ad2ab099e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:35.682557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b5f55-8faf-4983-b160-57ad2ab099e5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:35.682833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b5f55-8faf-4983-b160-57ad2ab099e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:35.683100 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b5f55-8faf-4983-b160-57ad2ab099e5 None admin] PATH_INFO: `/v3/projects/0cc6e70988094ce486a8b423cc6e1d04` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:35.684206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b5f55-8faf-4983-b160-57ad2ab099e5 None admin] RBAC: Authorizing `identity:get_project(project_id=0cc6e70988094ce486a8b423cc6e1d04)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:35.686274 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b5f55-8faf-4983-b160-57ad2ab099e5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:35.687909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 607/1199] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 08:28:35 2022] GET /identity/v3/projects/0cc6e70988094ce486a8b423cc6e1d04 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:35.694195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eb0d8-04c1-4772-9931-c86e4c92d457 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:28:35.711767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:35.791161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eb0d8-04c1-4772-9931-c86e4c92d457 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:28:35.825305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec9bbe-f9a7-4c43-b089-526d56dd1424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:28:35.825922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec9bbe-f9a7-4c43-b089-526d56dd1424 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:28:35.826218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec9bbe-f9a7-4c43-b089-526d56dd1424 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:28:35.826485 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec9bbe-f9a7-4c43-b089-526d56dd1424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:28:35.827193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec9bbe-f9a7-4c43-b089-526d56dd1424 None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=0cc6e70988094ce486a8b423cc6e1d04, user.id=57cb7c95f1234265a121885cd1cbfa0d) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:28:35.827492 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec9bbe-f9a7-4c43-b089-526d56dd1424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:28:35.828910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec9bbe-f9a7-4c43-b089-526d56dd1424 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:28:35.835058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 593/1200] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:28:35 2022] GET /identity/v3/role_assignments?user.id=57cb7c95f1234265a121885cd1cbfa0d&scope.project.id=0cc6e70988094ce486a8b423cc6e1d04&role.id=020bc02dca71485bab2312442adaa179 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:28:45.677786 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=56 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:28:45.832075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=60 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:29:54.928777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88aa90a-1e7f-4744-b08a-76eb605455ef None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:54.929394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88aa90a-1e7f-4744-b08a-76eb605455ef None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:54.929394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88aa90a-1e7f-4744-b08a-76eb605455ef None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:54.930927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 608/1201] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:29:54.937653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f7a18-2635-4946-8cac-bb683038e536 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:54.940295 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f7a18-2635-4946-8cac-bb683038e536 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:54.940295 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f7a18-2635-4946-8cac-bb683038e536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:54.957991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-189f7a18-2635-4946-8cac-bb683038e536 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:29:54.959247 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-189f7a18-2635-4946-8cac-bb683038e536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:29:55.033831 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 594/1202] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:29:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 08:29:55.308634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88aa90a-1e7f-4744-b08a-76eb605455ef None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:29:55.423817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88aa90a-1e7f-4744-b08a-76eb605455ef None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:29:55.458793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fd4bd-dfdd-40b0-8f15-3e9e09e904bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:29:55.459422 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fd4bd-dfdd-40b0-8f15-3e9e09e904bd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:55.459700 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fd4bd-dfdd-40b0-8f15-3e9e09e904bd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:55.459972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fd4bd-dfdd-40b0-8f15-3e9e09e904bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:55.471976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206fd4bd-dfdd-40b0-8f15-3e9e09e904bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:29:55.473429 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206fd4bd-dfdd-40b0-8f15-3e9e09e904bd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-206fd4bd-dfdd-40b0-8f15-3e9e09e904bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:29:55.490433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:29:55.491721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:29:55.492701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:29:55.492701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:29:55.492701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:29:55.492701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 609/1203] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:29:55.498053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f7a18-2635-4946-8cac-bb683038e536 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:29:55.597795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f7a18-2635-4946-8cac-bb683038e536 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:29:55.627495 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:29:55.633277 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb37449-558a-4552-a460-43f84fc1b6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:29:55.633877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb37449-558a-4552-a460-43f84fc1b6fc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:55.634168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb37449-558a-4552-a460-43f84fc1b6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:55.634470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb37449-558a-4552-a460-43f84fc1b6fc None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:55.635064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb37449-558a-4552-a460-43f84fc1b6fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:29:55.635355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb37449-558a-4552-a460-43f84fc1b6fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:29:55.636726 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb37449-558a-4552-a460-43f84fc1b6fc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:29:55.647670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 595/1204] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:29:55.653867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fd4bd-dfdd-40b0-8f15-3e9e09e904bd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:29:55.707905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:29:55.754416 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fd4bd-dfdd-40b0-8f15-3e9e09e904bd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:29:55.789587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7567a-ca31-49dc-94e0-5a363409bff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:29:55.790305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7567a-ca31-49dc-94e0-5a363409bff6 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:55.790632 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7567a-ca31-49dc-94e0-5a363409bff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:55.790901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7567a-ca31-49dc-94e0-5a363409bff6 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:55.791543 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7567a-ca31-49dc-94e0-5a363409bff6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:29:55.792968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7567a-ca31-49dc-94e0-5a363409bff6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:29:55.994790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 610/1205] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 08:29:55 2022] POST /identity/v3/users => generated 251 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:29:58.247678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ce1f7-0a3a-4b88-b8d7-a35be3559cb1 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:58.248204 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ce1f7-0a3a-4b88-b8d7-a35be3559cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:58.248204 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ce1f7-0a3a-4b88-b8d7-a35be3559cb1 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:58.250042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 596/1206] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:29:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:29:58.255446 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9df628-8ce3-4cb3-8448-ac329c11cc29 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:58.255907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9df628-8ce3-4cb3-8448-ac329c11cc29 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:58.256032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9df628-8ce3-4cb3-8448-ac329c11cc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:58.274943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8c9df628-8ce3-4cb3-8448-ac329c11cc29 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:29:58.276136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8c9df628-8ce3-4cb3-8448-ac329c11cc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:29:58.348133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 611/1207] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:29:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:29:58.621189 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ce1f7-0a3a-4b88-b8d7-a35be3559cb1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:29:58.732824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ce1f7-0a3a-4b88-b8d7-a35be3559cb1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:29:58.768360 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929678f-ce61-4d89-8f1a-8d98242e66b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:29:58.768991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929678f-ce61-4d89-8f1a-8d98242e66b4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:58.769266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929678f-ce61-4d89-8f1a-8d98242e66b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:58.769535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929678f-ce61-4d89-8f1a-8d98242e66b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:58.781730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929678f-ce61-4d89-8f1a-8d98242e66b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9929678f-ce61-4d89-8f1a-8d98242e66b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:29:58.786578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:29:58.787692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:29:58.788565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:29:58.788565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:29:58.788565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:29:58.788565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:29:58.788565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:29:58.788565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:29:58.788565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:29:58.788565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:29:58.788565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:29:58.788565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:29:58.788565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:29:58.788565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:29:58.788565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 597/1208] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:29:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:29:58.793956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9df628-8ce3-4cb3-8448-ac329c11cc29 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:29:58.834434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:29:58.894522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9df628-8ce3-4cb3-8448-ac329c11cc29 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:29:58.929848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f8479-573a-4fdc-b14d-5aa3a661c742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:29:58.930475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f8479-573a-4fdc-b14d-5aa3a661c742 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:58.930750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f8479-573a-4fdc-b14d-5aa3a661c742 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:58.931013 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f8479-573a-4fdc-b14d-5aa3a661c742 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:58.931672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f8479-573a-4fdc-b14d-5aa3a661c742 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:29:58.931965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f8479-573a-4fdc-b14d-5aa3a661c742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:29:58.933327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f8479-573a-4fdc-b14d-5aa3a661c742 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:29:58.943071 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 612/1209] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:29:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:29:58.949201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929678f-ce61-4d89-8f1a-8d98242e66b4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:29:59.003447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:29:59.053408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929678f-ce61-4d89-8f1a-8d98242e66b4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:29:59.086425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8292b-0275-4d6d-91ec-faefb8b084ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:29:59.088257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8292b-0275-4d6d-91ec-faefb8b084ac None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:59.088257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8292b-0275-4d6d-91ec-faefb8b084ac None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:59.088257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8292b-0275-4d6d-91ec-faefb8b084ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:59.098549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8292b-0275-4d6d-91ec-faefb8b084ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:29:59.099990 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff8292b-0275-4d6d-91ec-faefb8b084ac None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-cff8292b-0275-4d6d-91ec-faefb8b084ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:29:59.114708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:29:59.115956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:29:59.117144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:29:59.117144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:29:59.117144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:29:59.117144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 598/1210] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:29:59.122285 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f8479-573a-4fdc-b14d-5aa3a661c742 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:29:59.226487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:29:59.229786 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f8479-573a-4fdc-b14d-5aa3a661c742 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:29:59.264368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c158a1-1458-4fdc-8989-3009246fe8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:29:59.265062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c158a1-1458-4fdc-8989-3009246fe8ed None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:59.265237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c158a1-1458-4fdc-8989-3009246fe8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:59.265508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c158a1-1458-4fdc-8989-3009246fe8ed None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:59.266260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c158a1-1458-4fdc-8989-3009246fe8ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:29:59.266563 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c158a1-1458-4fdc-8989-3009246fe8ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:29:59.267930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c158a1-1458-4fdc-8989-3009246fe8ed None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:29:59.280495 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 613/1211] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:29:59.286639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8292b-0275-4d6d-91ec-faefb8b084ac None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:29:59.390967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8292b-0275-4d6d-91ec-faefb8b084ac None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:29:59.392543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:29:59.424803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c60b5c-fb92-4914-aac9-f188296dbded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:29:59.425454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c60b5c-fb92-4914-aac9-f188296dbded None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:59.425725 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c60b5c-fb92-4914-aac9-f188296dbded None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:59.426064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c60b5c-fb92-4914-aac9-f188296dbded None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:59.434081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c60b5c-fb92-4914-aac9-f188296dbded None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:29:59.435708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c60b5c-fb92-4914-aac9-f188296dbded None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f4c60b5c-fb92-4914-aac9-f188296dbded None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:29:59.452350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:29:59.456271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:29:59.457484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:29:59.457484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:29:59.457484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:29:59.457484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:29:59.457484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:29:59.457484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:29:59.457484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:29:59.457484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:29:59.457484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 08:29:59.457484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:29:59.457484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 599/1212] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:29:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:29:59.459622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c158a1-1458-4fdc-8989-3009246fe8ed None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:29:59.567409 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c158a1-1458-4fdc-8989-3009246fe8ed None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:29:59.604997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218bd842-295a-44c0-9eed-8c1477cf905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:29:59.605627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bd842-295a-44c0-9eed-8c1477cf905c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:59.605901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bd842-295a-44c0-9eed-8c1477cf905c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:59.606433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bd842-295a-44c0-9eed-8c1477cf905c None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:59.607032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218bd842-295a-44c0-9eed-8c1477cf905c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:29:59.607331 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218bd842-295a-44c0-9eed-8c1477cf905c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:29:59.608984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218bd842-295a-44c0-9eed-8c1477cf905c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:29:59.630453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 614/1213] 10.222.0.44 () {56 vars in 1143 bytes} [Sun Sep 11 08:29:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:29:59.636686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c60b5c-fb92-4914-aac9-f188296dbded None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:29:59.756429 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c60b5c-fb92-4914-aac9-f188296dbded None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:29:59.797530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3823e49-7622-4bb3-86ad-b669876da08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:29:59.798133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3823e49-7622-4bb3-86ad-b669876da08c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:29:59.798665 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3823e49-7622-4bb3-86ad-b669876da08c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:29:59.799064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3823e49-7622-4bb3-86ad-b669876da08c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:29:59.855390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:29:59.903644 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3823e49-7622-4bb3-86ad-b669876da08c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:29:59.905195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3823e49-7622-4bb3-86ad-b669876da08c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:00.020104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 600/1214] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:29:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:30:00.026115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218bd842-295a-44c0-9eed-8c1477cf905c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:00.048332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:00.134467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218bd842-295a-44c0-9eed-8c1477cf905c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:00.169004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b80a4-a9c9-4e02-a590-591598bd956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:00.169676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b80a4-a9c9-4e02-a590-591598bd956f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:00.169943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b80a4-a9c9-4e02-a590-591598bd956f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:00.170235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b80a4-a9c9-4e02-a590-591598bd956f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:00.182054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b80a4-a9c9-4e02-a590-591598bd956f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:00.183639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b80a4-a9c9-4e02-a590-591598bd956f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-118b80a4-a9c9-4e02-a590-591598bd956f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:30:00.199794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:00.201083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:30:00.202423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:30:00.202423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:00.202423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:00.206378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 615/1215] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:00.207589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3823e49-7622-4bb3-86ad-b669876da08c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:00.275982 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:00.321799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3823e49-7622-4bb3-86ad-b669876da08c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:00.360852 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198979b-5560-41ee-9b59-0b8971fef3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:00.361764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198979b-5560-41ee-9b59-0b8971fef3cb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:00.362283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198979b-5560-41ee-9b59-0b8971fef3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:00.362761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198979b-5560-41ee-9b59-0b8971fef3cb None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:00.363404 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1198979b-5560-41ee-9b59-0b8971fef3cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:00.363711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1198979b-5560-41ee-9b59-0b8971fef3cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:00.365444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1198979b-5560-41ee-9b59-0b8971fef3cb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:00.377352 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 601/1216] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:30:00.383382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b80a4-a9c9-4e02-a590-591598bd956f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:00.469453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:00.507288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b80a4-a9c9-4e02-a590-591598bd956f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:00.555469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b764fa7-1bdd-42f6-a6d2-b879c8272e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:00.556139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b764fa7-1bdd-42f6-a6d2-b879c8272e00 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:00.556403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b764fa7-1bdd-42f6-a6d2-b879c8272e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:00.556675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b764fa7-1bdd-42f6-a6d2-b879c8272e00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:00.565540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b764fa7-1bdd-42f6-a6d2-b879c8272e00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:00.567803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b764fa7-1bdd-42f6-a6d2-b879c8272e00 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1b764fa7-1bdd-42f6-a6d2-b879c8272e00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:30:00.580310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:00.587607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:00.588744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:30:00.588744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:30:00.588744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:30:00.588744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:30:00.588744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:30:00.588744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:30:00.588744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:30:00.588744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:00.588744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 616/1217] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:30:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:00.589313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198979b-5560-41ee-9b59-0b8971fef3cb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:00.692752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198979b-5560-41ee-9b59-0b8971fef3cb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:00.716184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:00.728134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c16c26-1eb6-4f13-a56e-256144a5273d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:00.728675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c16c26-1eb6-4f13-a56e-256144a5273d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:00.728953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c16c26-1eb6-4f13-a56e-256144a5273d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:00.729223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c16c26-1eb6-4f13-a56e-256144a5273d None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:00.729912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c16c26-1eb6-4f13-a56e-256144a5273d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:00.730254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c16c26-1eb6-4f13-a56e-256144a5273d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:00.731752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c16c26-1eb6-4f13-a56e-256144a5273d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:00.743896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 602/1218] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:30:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:00.750676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b764fa7-1bdd-42f6-a6d2-b879c8272e00 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:00.852161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b764fa7-1bdd-42f6-a6d2-b879c8272e00 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:00.880822 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:00.886653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c525b1-6a04-4caa-ab97-08c23cf11367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:00.887248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c525b1-6a04-4caa-ab97-08c23cf11367 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:00.887520 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c525b1-6a04-4caa-ab97-08c23cf11367 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:00.887782 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c525b1-6a04-4caa-ab97-08c23cf11367 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:00.888491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c525b1-6a04-4caa-ab97-08c23cf11367 None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=8d03db170fb6460794e32669456bcdaa) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:00.888863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c525b1-6a04-4caa-ab97-08c23cf11367 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:00.890981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c525b1-6a04-4caa-ab97-08c23cf11367 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:00.896398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 617/1219] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:30:00 2022] GET /identity/v3/role_assignments?user.id=8d03db170fb6460794e32669456bcdaa&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:03.067543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077c30d-2d87-4272-8e89-4dd8652a22e5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:03.068272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077c30d-2d87-4272-8e89-4dd8652a22e5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:03.068272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077c30d-2d87-4272-8e89-4dd8652a22e5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:03.069480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 603/1220] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:30:03.074916 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed310d-7fe0-4bbe-a09f-3f0f3f13d977 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:03.075381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed310d-7fe0-4bbe-a09f-3f0f3f13d977 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:03.075481 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed310d-7fe0-4bbe-a09f-3f0f3f13d977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:03.094855 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b1ed310d-7fe0-4bbe-a09f-3f0f3f13d977 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:30:03.096072 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b1ed310d-7fe0-4bbe-a09f-3f0f3f13d977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:30:03.164952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 618/1221] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:30:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:30:03.437342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077c30d-2d87-4272-8e89-4dd8652a22e5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:03.547053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077c30d-2d87-4272-8e89-4dd8652a22e5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:03.581030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce71c7-6101-42de-8c15-4e391547d428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:03.581671 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce71c7-6101-42de-8c15-4e391547d428 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:03.581945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce71c7-6101-42de-8c15-4e391547d428 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:03.582242 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce71c7-6101-42de-8c15-4e391547d428 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:03.595894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce71c7-6101-42de-8c15-4e391547d428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-69ce71c7-6101-42de-8c15-4e391547d428 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:30:03.600962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:03.602103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:03.603361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:03.603361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:03.603361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:03.603361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:03.603361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:30:03.603361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:30:03.603361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:30:03.603361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:30:03.603361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:30:03.603361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:30:03.603361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:30:03.603361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:03.603361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 604/1222] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:30:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:03.608447 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed310d-7fe0-4bbe-a09f-3f0f3f13d977 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:03.712451 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed310d-7fe0-4bbe-a09f-3f0f3f13d977 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:03.737326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:03.747953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8231c-3058-43db-8337-47dc3e8fb907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:03.749086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8231c-3058-43db-8337-47dc3e8fb907 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:03.749360 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8231c-3058-43db-8337-47dc3e8fb907 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:03.749632 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8231c-3058-43db-8337-47dc3e8fb907 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:03.750328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8231c-3058-43db-8337-47dc3e8fb907 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:03.750632 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8231c-3058-43db-8337-47dc3e8fb907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:03.752005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8231c-3058-43db-8337-47dc3e8fb907 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:03.760576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 619/1223] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:30:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:03.767453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce71c7-6101-42de-8c15-4e391547d428 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:03.812909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:03.871310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce71c7-6101-42de-8c15-4e391547d428 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:03.906752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a2d2a-e88b-468f-a054-27d5a09d4765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:03.907377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a2d2a-e88b-468f-a054-27d5a09d4765 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:03.907651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a2d2a-e88b-468f-a054-27d5a09d4765 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:03.907914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a2d2a-e88b-468f-a054-27d5a09d4765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:03.917904 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a2d2a-e88b-468f-a054-27d5a09d4765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:03.919391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a2d2a-e88b-468f-a054-27d5a09d4765 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d42a2d2a-e88b-468f-a054-27d5a09d4765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:30:03.934199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:03.935303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:30:03.937264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:30:03.937264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:03.937264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:03.937264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 605/1224] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:03.942236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8231c-3058-43db-8337-47dc3e8fb907 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:04.048740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8231c-3058-43db-8337-47dc3e8fb907 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:04.082743 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebf72f-e3c4-4cae-8a47-c76394016d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:04.083437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebf72f-e3c4-4cae-8a47-c76394016d38 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:04.083626 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebf72f-e3c4-4cae-8a47-c76394016d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:04.083896 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebf72f-e3c4-4cae-8a47-c76394016d38 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:04.084547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebf72f-e3c4-4cae-8a47-c76394016d38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:04.084849 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebf72f-e3c4-4cae-8a47-c76394016d38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:04.086281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebf72f-e3c4-4cae-8a47-c76394016d38 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:04.101591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 620/1225] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:04.109305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a2d2a-e88b-468f-a054-27d5a09d4765 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:04.209697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:04.216518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a2d2a-e88b-468f-a054-27d5a09d4765 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:04.255319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188b212-b6ed-47b3-8a8c-c72bd6750e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:04.255951 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188b212-b6ed-47b3-8a8c-c72bd6750e03 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:04.256232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188b212-b6ed-47b3-8a8c-c72bd6750e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:04.256506 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188b212-b6ed-47b3-8a8c-c72bd6750e03 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:04.264662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188b212-b6ed-47b3-8a8c-c72bd6750e03 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:04.266100 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188b212-b6ed-47b3-8a8c-c72bd6750e03 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4188b212-b6ed-47b3-8a8c-c72bd6750e03 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:30:04.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:04.278616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:04.279888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:04.279888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:04.279888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:30:04.279888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:30:04.279888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:30:04.279888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:30:04.279888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:30:04.279888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:30:04.279888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 08:30:04.279888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:04.282309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 606/1226] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:04.284741 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebf72f-e3c4-4cae-8a47-c76394016d38 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:04.338974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:04.391077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebf72f-e3c4-4cae-8a47-c76394016d38 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:04.426844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad7dc6-f2ae-4b2e-998a-63e65d0a57f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:04.427453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad7dc6-f2ae-4b2e-998a-63e65d0a57f3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:04.427731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad7dc6-f2ae-4b2e-998a-63e65d0a57f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:04.428003 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad7dc6-f2ae-4b2e-998a-63e65d0a57f3 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:04.428726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad7dc6-f2ae-4b2e-998a-63e65d0a57f3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:04.429140 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad7dc6-f2ae-4b2e-998a-63e65d0a57f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:04.430600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad7dc6-f2ae-4b2e-998a-63e65d0a57f3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:04.448796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 621/1227] 10.222.0.44 () {56 vars in 1143 bytes} [Sun Sep 11 08:30:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:04.454779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188b212-b6ed-47b3-8a8c-c72bd6750e03 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:04.554997 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188b212-b6ed-47b3-8a8c-c72bd6750e03 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:04.587678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c068f2-b01b-4745-867f-f3a6a179e92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:04.588348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c068f2-b01b-4745-867f-f3a6a179e92b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:04.588625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c068f2-b01b-4745-867f-f3a6a179e92b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:04.588891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c068f2-b01b-4745-867f-f3a6a179e92b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:04.600104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c068f2-b01b-4745-867f-f3a6a179e92b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:04.601527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c068f2-b01b-4745-867f-f3a6a179e92b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-06c068f2-b01b-4745-867f-f3a6a179e92b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:30:04.616130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:04.623857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:30:04.624960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:30:04.624960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:04.624960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:04.624960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 607/1228] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:04.625318 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad7dc6-f2ae-4b2e-998a-63e65d0a57f3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:04.724262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad7dc6-f2ae-4b2e-998a-63e65d0a57f3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:04.726751 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:04.759956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50025a3-6adf-4e3c-ad81-e1ccab032046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:04.760592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50025a3-6adf-4e3c-ad81-e1ccab032046 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:04.760869 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50025a3-6adf-4e3c-ad81-e1ccab032046 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:04.761136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50025a3-6adf-4e3c-ad81-e1ccab032046 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:04.761768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50025a3-6adf-4e3c-ad81-e1ccab032046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:04.762073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50025a3-6adf-4e3c-ad81-e1ccab032046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:04.763490 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50025a3-6adf-4e3c-ad81-e1ccab032046 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:04.774669 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 622/1229] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:04.780415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c068f2-b01b-4745-867f-f3a6a179e92b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:04.790806 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:04.882704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c068f2-b01b-4745-867f-f3a6a179e92b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:04.917114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2edcdcc-8121-42de-b2b9-a3a13a10a610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:04.917738 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edcdcc-8121-42de-b2b9-a3a13a10a610 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:04.918017 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edcdcc-8121-42de-b2b9-a3a13a10a610 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:04.918320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edcdcc-8121-42de-b2b9-a3a13a10a610 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:04.925450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2edcdcc-8121-42de-b2b9-a3a13a10a610 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:04.927063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2edcdcc-8121-42de-b2b9-a3a13a10a610 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e2edcdcc-8121-42de-b2b9-a3a13a10a610 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:30:04.939050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:04.940004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:04.941279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:30:04.941279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:30:04.941279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:30:04.941279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:30:04.941279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:30:04.941279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:30:04.941279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:30:04.941279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:04.941279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 608/1230] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:30:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:04.946557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50025a3-6adf-4e3c-ad81-e1ccab032046 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:05.049274 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50025a3-6adf-4e3c-ad81-e1ccab032046 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:05.082882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1324ec9-b1af-462d-80ee-c8459ca4d9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:05.083510 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324ec9-b1af-462d-80ee-c8459ca4d9fa None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:05.083759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324ec9-b1af-462d-80ee-c8459ca4d9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:05.084022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324ec9-b1af-462d-80ee-c8459ca4d9fa None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:05.084693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1324ec9-b1af-462d-80ee-c8459ca4d9fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:05.084990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1324ec9-b1af-462d-80ee-c8459ca4d9fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:05.086494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1324ec9-b1af-462d-80ee-c8459ca4d9fa None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:05.098677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 623/1231] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:30:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:30:05.104696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2edcdcc-8121-42de-b2b9-a3a13a10a610 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:05.171285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:05.205120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2edcdcc-8121-42de-b2b9-a3a13a10a610 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:05.239793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57616d1-9c08-42ce-ad36-ee3ec5e4b308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:05.240507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57616d1-9c08-42ce-ad36-ee3ec5e4b308 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:05.240778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57616d1-9c08-42ce-ad36-ee3ec5e4b308 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:05.241039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57616d1-9c08-42ce-ad36-ee3ec5e4b308 None admin] PATH_INFO: `/v3/projects/811863e9c94d44a39647800f306505f0/users/8d03db170fb6460794e32669456bcdaa/roles/a76c253615fd43a09d6dc924dbe16553` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:05.255297 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57616d1-9c08-42ce-ad36-ee3ec5e4b308 None admin] RBAC: Authorizing `identity:create_grant(project_id=811863e9c94d44a39647800f306505f0, user_id=8d03db170fb6460794e32669456bcdaa, role_id=a76c253615fd43a09d6dc924dbe16553)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:05.256700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57616d1-9c08-42ce-ad36-ee3ec5e4b308 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:05.328138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 609/1232] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:30:05 2022] PUT /identity/v3/projects/811863e9c94d44a39647800f306505f0/users/8d03db170fb6460794e32669456bcdaa/roles/a76c253615fd43a09d6dc924dbe16553 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:30:07.471380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b5eaf-6b52-477f-9add-929aeb56c915 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:07.473674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b5eaf-6b52-477f-9add-929aeb56c915 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:07.473674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b5eaf-6b52-477f-9add-929aeb56c915 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:07.473674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 624/1233] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:30:07.478876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d56e5f-c5f5-4be8-a900-c08b95c0ea48 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:07.479477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d56e5f-c5f5-4be8-a900-c08b95c0ea48 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:07.479477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d56e5f-c5f5-4be8-a900-c08b95c0ea48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:07.499134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e0d56e5f-c5f5-4be8-a900-c08b95c0ea48 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:30:07.502142 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e0d56e5f-c5f5-4be8-a900-c08b95c0ea48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:30:07.571922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 610/1234] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:30:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:30:07.853176 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b5eaf-6b52-477f-9add-929aeb56c915 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:07.885483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:07.965684 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b5eaf-6b52-477f-9add-929aeb56c915 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:08.002267 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb4fcc-165e-4e9e-b45e-8dcf876fcab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:08.002895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb4fcc-165e-4e9e-b45e-8dcf876fcab1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:08.003175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb4fcc-165e-4e9e-b45e-8dcf876fcab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:08.003451 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb4fcc-165e-4e9e-b45e-8dcf876fcab1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:08.013630 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb4fcc-165e-4e9e-b45e-8dcf876fcab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b2eb4fcc-165e-4e9e-b45e-8dcf876fcab1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:30:08.019033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:08.020233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:08.021368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:08.021368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:08.021368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:08.021368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:08.021368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:30:08.021368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:30:08.021368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:30:08.021368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:30:08.021368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:30:08.021368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:30:08.021368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 08:30:08.021368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:08.021368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 625/1235] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 08:30:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:30:08.027217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d56e5f-c5f5-4be8-a900-c08b95c0ea48 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:08.072874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:08.125863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d56e5f-c5f5-4be8-a900-c08b95c0ea48 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:08.163997 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617a0d2-917d-4986-ad1d-316742a41c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:08.164595 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617a0d2-917d-4986-ad1d-316742a41c93 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:08.164865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617a0d2-917d-4986-ad1d-316742a41c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:08.165127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617a0d2-917d-4986-ad1d-316742a41c93 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:08.165803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617a0d2-917d-4986-ad1d-316742a41c93 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:08.166214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617a0d2-917d-4986-ad1d-316742a41c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:08.167610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617a0d2-917d-4986-ad1d-316742a41c93 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:08.176299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 611/1236] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 08:30:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:08.182461 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb4fcc-165e-4e9e-b45e-8dcf876fcab1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:08.269419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:08.287508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb4fcc-165e-4e9e-b45e-8dcf876fcab1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:08.321747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642f949-24f9-454b-95a3-7a8f9930155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:08.322404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642f949-24f9-454b-95a3-7a8f9930155d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:08.322683 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642f949-24f9-454b-95a3-7a8f9930155d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:08.322954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642f949-24f9-454b-95a3-7a8f9930155d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:08.334654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642f949-24f9-454b-95a3-7a8f9930155d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:08.336154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642f949-24f9-454b-95a3-7a8f9930155d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b642f949-24f9-454b-95a3-7a8f9930155d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:30:08.352219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:08.353452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:30:08.354618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:30:08.354618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:08.354618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:08.358463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 626/1237] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:08.359292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617a0d2-917d-4986-ad1d-316742a41c93 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:08.459814 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617a0d2-917d-4986-ad1d-316742a41c93 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:08.469321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:08.494115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37478a30-9315-42e3-bf79-24c5e1b1bf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:08.495471 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37478a30-9315-42e3-bf79-24c5e1b1bf41 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:08.495471 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37478a30-9315-42e3-bf79-24c5e1b1bf41 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:08.495471 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37478a30-9315-42e3-bf79-24c5e1b1bf41 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:08.496024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37478a30-9315-42e3-bf79-24c5e1b1bf41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:08.496366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37478a30-9315-42e3-bf79-24c5e1b1bf41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:08.497723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37478a30-9315-42e3-bf79-24c5e1b1bf41 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:08.511135 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 612/1238] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:30:08.517075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642f949-24f9-454b-95a3-7a8f9930155d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:08.625106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642f949-24f9-454b-95a3-7a8f9930155d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:08.658990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e367bf-b4f5-4a21-b8b6-91bc600c9bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:08.659611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e367bf-b4f5-4a21-b8b6-91bc600c9bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:08.659926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e367bf-b4f5-4a21-b8b6-91bc600c9bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:08.660157 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e367bf-b4f5-4a21-b8b6-91bc600c9bd9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:08.662880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:08.668058 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e367bf-b4f5-4a21-b8b6-91bc600c9bd9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:08.669499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e367bf-b4f5-4a21-b8b6-91bc600c9bd9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d7e367bf-b4f5-4a21-b8b6-91bc600c9bd9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:30:08.681776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:08.686675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:08.687682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:08.687682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:08.687682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:30:08.687682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:30:08.687682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:30:08.687682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:30:08.687682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:30:08.687682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:30:08.687682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 08:30:08.687682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:08.687682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 627/1239] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:08.689912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37478a30-9315-42e3-bf79-24c5e1b1bf41 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:08.795952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37478a30-9315-42e3-bf79-24c5e1b1bf41 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:08.835811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2721f-920c-4df9-9f89-abb4cbeb0daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:08.836432 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2721f-920c-4df9-9f89-abb4cbeb0daf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:08.836706 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2721f-920c-4df9-9f89-abb4cbeb0daf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:08.836972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2721f-920c-4df9-9f89-abb4cbeb0daf None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:08.837722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2721f-920c-4df9-9f89-abb4cbeb0daf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:08.838022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2721f-920c-4df9-9f89-abb4cbeb0daf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:08.839635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2721f-920c-4df9-9f89-abb4cbeb0daf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:08.863755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 613/1240] 10.222.0.44 () {56 vars in 1143 bytes} [Sun Sep 11 08:30:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:30:08.870689 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e367bf-b4f5-4a21-b8b6-91bc600c9bd9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:08.990774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e367bf-b4f5-4a21-b8b6-91bc600c9bd9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:09.026235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c686d4-d083-4db2-a94f-cacb997021eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:09.026868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c686d4-d083-4db2-a94f-cacb997021eb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:09.027182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c686d4-d083-4db2-a94f-cacb997021eb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:09.027456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c686d4-d083-4db2-a94f-cacb997021eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:09.090615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:09.097188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c686d4-d083-4db2-a94f-cacb997021eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:09.098690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c686d4-d083-4db2-a94f-cacb997021eb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:09.194656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 628/1241] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:30:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 08:30:09.200635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2721f-920c-4df9-9f89-abb4cbeb0daf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:09.229979 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:09.301993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2721f-920c-4df9-9f89-abb4cbeb0daf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:09.336658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fe707-4680-4958-97b9-b863c0d54ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:09.337276 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fe707-4680-4958-97b9-b863c0d54ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:09.337547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fe707-4680-4958-97b9-b863c0d54ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:09.337812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fe707-4680-4958-97b9-b863c0d54ca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:09.348411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fe707-4680-4958-97b9-b863c0d54ca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:09.349765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fe707-4680-4958-97b9-b863c0d54ca2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-697fe707-4680-4958-97b9-b863c0d54ca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:30:09.365292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:09.366553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:30:09.367757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:30:09.367757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:09.367757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:09.367757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 614/1242] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:09.372414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c686d4-d083-4db2-a94f-cacb997021eb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:09.465360 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:09.471328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c686d4-d083-4db2-a94f-cacb997021eb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:09.505503 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad97d95-de71-44ac-9fae-7d823cc10495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:09.506113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97d95-de71-44ac-9fae-7d823cc10495 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:09.506495 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97d95-de71-44ac-9fae-7d823cc10495 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:09.506726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97d95-de71-44ac-9fae-7d823cc10495 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:09.507389 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad97d95-de71-44ac-9fae-7d823cc10495 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:09.507690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad97d95-de71-44ac-9fae-7d823cc10495 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:09.509044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad97d95-de71-44ac-9fae-7d823cc10495 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:09.520759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 629/1243] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:09.526512 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fe707-4680-4958-97b9-b863c0d54ca2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:09.605069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:09.627205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fe707-4680-4958-97b9-b863c0d54ca2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:09.660005 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682b841-1c07-4da7-95e0-2f0172ccc200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:09.660604 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682b841-1c07-4da7-95e0-2f0172ccc200 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:09.660884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682b841-1c07-4da7-95e0-2f0172ccc200 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:09.661150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682b841-1c07-4da7-95e0-2f0172ccc200 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:09.667985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682b841-1c07-4da7-95e0-2f0172ccc200 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:09.669468 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682b841-1c07-4da7-95e0-2f0172ccc200 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c682b841-1c07-4da7-95e0-2f0172ccc200 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:30:09.682077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:09.683193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:09.684116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:30:09.684116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:30:09.684116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:30:09.684116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:30:09.684116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:30:09.684116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:30:09.684116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:30:09.684116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:09.684116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 615/1244] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:30:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:09.689357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad97d95-de71-44ac-9fae-7d823cc10495 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:09.793027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad97d95-de71-44ac-9fae-7d823cc10495 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:09.826787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b2de4-0c7b-451c-bbc0-124140ae9e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:09.827467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b2de4-0c7b-451c-bbc0-124140ae9e0c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:09.827749 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b2de4-0c7b-451c-bbc0-124140ae9e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:09.828017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b2de4-0c7b-451c-bbc0-124140ae9e0c None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:09.828701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b2de4-0c7b-451c-bbc0-124140ae9e0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:09.829003 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b2de4-0c7b-451c-bbc0-124140ae9e0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:09.830452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b2de4-0c7b-451c-bbc0-124140ae9e0c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:09.843041 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 630/1245] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:30:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:09.849557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682b841-1c07-4da7-95e0-2f0172ccc200 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:09.949849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682b841-1c07-4da7-95e0-2f0172ccc200 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:09.984254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323007c4-1229-4586-9036-80c20dd766fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:09.984853 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323007c4-1229-4586-9036-80c20dd766fe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:09.985124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323007c4-1229-4586-9036-80c20dd766fe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:09.985392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323007c4-1229-4586-9036-80c20dd766fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:09.986183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323007c4-1229-4586-9036-80c20dd766fe None admin] RBAC: Adding query filter params (role.id=a76c253615fd43a09d6dc924dbe16553, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=8d03db170fb6460794e32669456bcdaa) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:09.991080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323007c4-1229-4586-9036-80c20dd766fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:09.991080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323007c4-1229-4586-9036-80c20dd766fe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:09.993730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 616/1246] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:30:09 2022] GET /identity/v3/role_assignments?user.id=8d03db170fb6460794e32669456bcdaa&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=a76c253615fd43a09d6dc924dbe16553 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:12.236518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc306363-84a9-4f83-a7a4-640cdf98228a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:12.237058 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc306363-84a9-4f83-a7a4-640cdf98228a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:12.237058 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc306363-84a9-4f83-a7a4-640cdf98228a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:12.238340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 631/1247] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:30:12.245144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d36c66-6a6f-4fb4-a78c-7a1dca5c0d94 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:12.245602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d36c66-6a6f-4fb4-a78c-7a1dca5c0d94 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:12.245715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d36c66-6a6f-4fb4-a78c-7a1dca5c0d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:12.252093 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:12.265771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-63d36c66-6a6f-4fb4-a78c-7a1dca5c0d94 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:30:12.267006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-63d36c66-6a6f-4fb4-a78c-7a1dca5c0d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:30:12.338753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 617/1248] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:30:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:30:12.611086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc306363-84a9-4f83-a7a4-640cdf98228a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:12.633790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:12.727261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc306363-84a9-4f83-a7a4-640cdf98228a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:12.765698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcb839-23d9-4003-8ca1-0da2deb69b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:12.766353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcb839-23d9-4003-8ca1-0da2deb69b08 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:12.766629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcb839-23d9-4003-8ca1-0da2deb69b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:12.766903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcb839-23d9-4003-8ca1-0da2deb69b08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:12.778945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcb839-23d9-4003-8ca1-0da2deb69b08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b6bcb839-23d9-4003-8ca1-0da2deb69b08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:30:12.783656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:12.784906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:12.786205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:12.786205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:12.786205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:12.786205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:12.786205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:30:12.786205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:30:12.786205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:30:12.786205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:30:12.786205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:30:12.786205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:30:12.786205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:30:12.786205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:12.786205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 632/1249] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:30:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:12.791092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d36c66-6a6f-4fb4-a78c-7a1dca5c0d94 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:12.896820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d36c66-6a6f-4fb4-a78c-7a1dca5c0d94 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:12.936956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:12.942165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd4fdd-de3b-4838-818f-01dd314578a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:12.942803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd4fdd-de3b-4838-818f-01dd314578a1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:12.943061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd4fdd-de3b-4838-818f-01dd314578a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:12.943326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd4fdd-de3b-4838-818f-01dd314578a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:12.943994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd4fdd-de3b-4838-818f-01dd314578a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:12.944327 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd4fdd-de3b-4838-818f-01dd314578a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:12.945770 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd4fdd-de3b-4838-818f-01dd314578a1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:12.955408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 618/1250] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:30:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:12.962527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcb839-23d9-4003-8ca1-0da2deb69b08 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:13.057814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:13.081182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcb839-23d9-4003-8ca1-0da2deb69b08 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:13.114351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a0b22-cd10-45cb-9a6c-dd0054f13c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:13.114989 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a0b22-cd10-45cb-9a6c-dd0054f13c9c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:13.115246 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a0b22-cd10-45cb-9a6c-dd0054f13c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:13.115509 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a0b22-cd10-45cb-9a6c-dd0054f13c9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:13.126662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a0b22-cd10-45cb-9a6c-dd0054f13c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:13.128057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a0b22-cd10-45cb-9a6c-dd0054f13c9c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ea9a0b22-cd10-45cb-9a6c-dd0054f13c9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:30:13.142457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:13.143601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:30:13.144575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:30:13.144575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:13.144575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:13.144575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 633/1251] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:30:13.149910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd4fdd-de3b-4838-818f-01dd314578a1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:13.270002 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd4fdd-de3b-4838-818f-01dd314578a1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:13.319684 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee826e6-6280-4c08-8bd2-8ce0068caac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:13.320578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee826e6-6280-4c08-8bd2-8ce0068caac0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:13.320996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee826e6-6280-4c08-8bd2-8ce0068caac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:13.321400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee826e6-6280-4c08-8bd2-8ce0068caac0 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:13.322424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee826e6-6280-4c08-8bd2-8ce0068caac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:13.322778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee826e6-6280-4c08-8bd2-8ce0068caac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:13.324842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee826e6-6280-4c08-8bd2-8ce0068caac0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:13.337892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 619/1252] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:13.344392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a0b22-cd10-45cb-9a6c-dd0054f13c9c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:13.450413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a0b22-cd10-45cb-9a6c-dd0054f13c9c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:13.477758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:13.486177 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d5db3-d91a-46d2-bb63-84756b57b3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:13.486805 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d5db3-d91a-46d2-bb63-84756b57b3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:13.487076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d5db3-d91a-46d2-bb63-84756b57b3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:13.487363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d5db3-d91a-46d2-bb63-84756b57b3a5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:13.495001 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d5db3-d91a-46d2-bb63-84756b57b3a5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:13.496436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d5db3-d91a-46d2-bb63-84756b57b3a5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a17d5db3-d91a-46d2-bb63-84756b57b3a5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:30:13.508579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:13.509707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:13.510792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:13.510792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:13.510792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:30:13.510792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:30:13.510792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:30:13.510792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:30:13.510792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:30:13.510792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:30:13.510792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 08:30:13.510792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:13.510792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 634/1253] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:13.516776 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee826e6-6280-4c08-8bd2-8ce0068caac0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:13.573555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:13.618762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee826e6-6280-4c08-8bd2-8ce0068caac0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:13.659573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e75b6-985c-4fe0-bfbc-4bca28e1dcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:13.660182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e75b6-985c-4fe0-bfbc-4bca28e1dcbb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:13.660472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e75b6-985c-4fe0-bfbc-4bca28e1dcbb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:13.660991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e75b6-985c-4fe0-bfbc-4bca28e1dcbb None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:13.662403 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e75b6-985c-4fe0-bfbc-4bca28e1dcbb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:13.662518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e75b6-985c-4fe0-bfbc-4bca28e1dcbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:13.663977 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e75b6-985c-4fe0-bfbc-4bca28e1dcbb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:13.682884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 620/1254] 10.222.0.44 () {56 vars in 1143 bytes} [Sun Sep 11 08:30:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:13.689120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d5db3-d91a-46d2-bb63-84756b57b3a5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:13.803455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d5db3-d91a-46d2-bb63-84756b57b3a5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:13.840654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b447c-178a-43f6-94e8-0e3b27d44d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:13.841263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b447c-178a-43f6-94e8-0e3b27d44d66 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:13.841540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b447c-178a-43f6-94e8-0e3b27d44d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:13.842448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b447c-178a-43f6-94e8-0e3b27d44d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:13.900863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:13.915917 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b447c-178a-43f6-94e8-0e3b27d44d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:13.917530 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b447c-178a-43f6-94e8-0e3b27d44d66 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:14.021468 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 635/1255] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:30:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 08:30:14.027459 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e75b6-985c-4fe0-bfbc-4bca28e1dcbb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:14.127907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e75b6-985c-4fe0-bfbc-4bca28e1dcbb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:14.141635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:14.163661 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0beb4-3921-432f-b9a9-458ef535c771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:14.164284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0beb4-3921-432f-b9a9-458ef535c771 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:14.164558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0beb4-3921-432f-b9a9-458ef535c771 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:14.164843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0beb4-3921-432f-b9a9-458ef535c771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:14.175049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0beb4-3921-432f-b9a9-458ef535c771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:14.176488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0beb4-3921-432f-b9a9-458ef535c771 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5dc0beb4-3921-432f-b9a9-458ef535c771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:30:14.192676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:14.197413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:30:14.198544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:30:14.198544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:14.198544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:14.198544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 621/1256] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:14.199637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b447c-178a-43f6-94e8-0e3b27d44d66 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:14.289959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:14.302529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b447c-178a-43f6-94e8-0e3b27d44d66 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:14.340286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f3590-fe0f-4018-80ed-a194ccad76b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:14.340882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f3590-fe0f-4018-80ed-a194ccad76b4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:14.341159 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f3590-fe0f-4018-80ed-a194ccad76b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:14.341429 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f3590-fe0f-4018-80ed-a194ccad76b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:14.342091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f3590-fe0f-4018-80ed-a194ccad76b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:14.342418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f3590-fe0f-4018-80ed-a194ccad76b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:14.343770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f3590-fe0f-4018-80ed-a194ccad76b4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:14.355287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 636/1257] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:14.361146 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0beb4-3921-432f-b9a9-458ef535c771 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:14.463832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0beb4-3921-432f-b9a9-458ef535c771 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:14.496743 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d3942-2dd7-4418-9cd3-9bf105a2046a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:14.497352 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d3942-2dd7-4418-9cd3-9bf105a2046a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:14.497631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d3942-2dd7-4418-9cd3-9bf105a2046a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:14.498469 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d3942-2dd7-4418-9cd3-9bf105a2046a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:14.504881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d3942-2dd7-4418-9cd3-9bf105a2046a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:14.506398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d3942-2dd7-4418-9cd3-9bf105a2046a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f97d3942-2dd7-4418-9cd3-9bf105a2046a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:30:14.517607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:14.518621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:14.519523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:30:14.519523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:30:14.519523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:30:14.519523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:30:14.519523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:30:14.519523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:30:14.519523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:30:14.519523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:14.519523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 622/1258] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:30:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:14.524662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f3590-fe0f-4018-80ed-a194ccad76b4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:14.628199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f3590-fe0f-4018-80ed-a194ccad76b4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:14.662485 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4c505-f238-4e8e-a9a3-6b5a63b64991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:14.663062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4c505-f238-4e8e-a9a3-6b5a63b64991 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:14.663366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4c505-f238-4e8e-a9a3-6b5a63b64991 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:14.663631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4c505-f238-4e8e-a9a3-6b5a63b64991 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:14.664307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4c505-f238-4e8e-a9a3-6b5a63b64991 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:14.664607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4c505-f238-4e8e-a9a3-6b5a63b64991 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:14.666137 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4c505-f238-4e8e-a9a3-6b5a63b64991 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:14.678660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 637/1259] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:30:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:14.684820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d3942-2dd7-4418-9cd3-9bf105a2046a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:14.712193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:14.787477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d3942-2dd7-4418-9cd3-9bf105a2046a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:14.820751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969d64c-6c2a-4fe7-bfa6-d7a6861c9651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:14.821365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969d64c-6c2a-4fe7-bfa6-d7a6861c9651 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:14.821629 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969d64c-6c2a-4fe7-bfa6-d7a6861c9651 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:14.821893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969d64c-6c2a-4fe7-bfa6-d7a6861c9651 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:14.822648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969d64c-6c2a-4fe7-bfa6-d7a6861c9651 None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=8d03db170fb6460794e32669456bcdaa) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:14.822957 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969d64c-6c2a-4fe7-bfa6-d7a6861c9651 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:14.824378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969d64c-6c2a-4fe7-bfa6-d7a6861c9651 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:14.830192 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 623/1260] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:30:14 2022] GET /identity/v3/role_assignments?user.id=8d03db170fb6460794e32669456bcdaa&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=020bc02dca71485bab2312442adaa179 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:17.019062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea672af-a35e-4629-bb16-5f4773be2ccf None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:17.019620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea672af-a35e-4629-bb16-5f4773be2ccf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:17.019620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea672af-a35e-4629-bb16-5f4773be2ccf None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:17.020957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 638/1261] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:30:17.026385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74b3b9-66f5-4a5e-a7b6-1cda391059fa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:17.026822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74b3b9-66f5-4a5e-a7b6-1cda391059fa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:17.026956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74b3b9-66f5-4a5e-a7b6-1cda391059fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:17.045889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3f74b3b9-66f5-4a5e-a7b6-1cda391059fa None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:30:17.047146 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3f74b3b9-66f5-4a5e-a7b6-1cda391059fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:30:17.116596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 624/1262] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:30:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:30:17.391817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea672af-a35e-4629-bb16-5f4773be2ccf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:17.403407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:17.508532 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea672af-a35e-4629-bb16-5f4773be2ccf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:17.543676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3b16d-54d1-478d-a9f4-fc05e63432b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:17.544318 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3b16d-54d1-478d-a9f4-fc05e63432b1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:17.544779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3b16d-54d1-478d-a9f4-fc05e63432b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:17.544887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3b16d-54d1-478d-a9f4-fc05e63432b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:17.567487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3b16d-54d1-478d-a9f4-fc05e63432b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ccd3b16d-54d1-478d-a9f4-fc05e63432b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:30:17.577435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:17.583862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:17.584973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:17.584973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:17.584973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:17.584973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:17.584973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:30:17.584973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:30:17.584973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:30:17.584973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:30:17.584973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:30:17.584973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:30:17.584973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:30:17.584973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:17.584973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 639/1263] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:30:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:17.586380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74b3b9-66f5-4a5e-a7b6-1cda391059fa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:17.611569 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:17.712426 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74b3b9-66f5-4a5e-a7b6-1cda391059fa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:17.768991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac80ff-4985-4a2a-821e-6c86cb052d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:17.769820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac80ff-4985-4a2a-821e-6c86cb052d06 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:17.770094 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac80ff-4985-4a2a-821e-6c86cb052d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:17.770431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac80ff-4985-4a2a-821e-6c86cb052d06 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:17.771311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac80ff-4985-4a2a-821e-6c86cb052d06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:17.771756 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac80ff-4985-4a2a-821e-6c86cb052d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:17.773831 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac80ff-4985-4a2a-821e-6c86cb052d06 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:17.789273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 625/1264] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:30:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:17.795408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3b16d-54d1-478d-a9f4-fc05e63432b1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:17.896398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:17.930672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3b16d-54d1-478d-a9f4-fc05e63432b1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:17.966223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac6196-91e2-4e83-b0fe-1c2b91ca9648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:17.966814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6196-91e2-4e83-b0fe-1c2b91ca9648 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:17.967096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6196-91e2-4e83-b0fe-1c2b91ca9648 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:17.967364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6196-91e2-4e83-b0fe-1c2b91ca9648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:17.979266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac6196-91e2-4e83-b0fe-1c2b91ca9648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:17.980719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac6196-91e2-4e83-b0fe-1c2b91ca9648 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-4eac6196-91e2-4e83-b0fe-1c2b91ca9648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:30:17.995633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:17.996836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:30:17.998306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:30:17.998306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:17.998306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:17.998306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 640/1265] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:18.004156 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac80ff-4985-4a2a-821e-6c86cb052d06 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:18.099766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:18.119777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac80ff-4985-4a2a-821e-6c86cb052d06 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:18.153733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe474-cdaf-4006-9663-4eb9f9289992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:18.154918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe474-cdaf-4006-9663-4eb9f9289992 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:18.155193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe474-cdaf-4006-9663-4eb9f9289992 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:18.155455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe474-cdaf-4006-9663-4eb9f9289992 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:18.156097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afe474-cdaf-4006-9663-4eb9f9289992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:18.156395 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afe474-cdaf-4006-9663-4eb9f9289992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:18.157792 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afe474-cdaf-4006-9663-4eb9f9289992 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:18.171319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 626/1266] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:18.177206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac6196-91e2-4e83-b0fe-1c2b91ca9648 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:18.282514 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac6196-91e2-4e83-b0fe-1c2b91ca9648 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:18.303142 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:18.316949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d834e0-8fa2-4cd9-9f76-2bc7f80bbb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:18.317574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d834e0-8fa2-4cd9-9f76-2bc7f80bbb4f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:18.317850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d834e0-8fa2-4cd9-9f76-2bc7f80bbb4f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:18.318121 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d834e0-8fa2-4cd9-9f76-2bc7f80bbb4f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:18.325638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d834e0-8fa2-4cd9-9f76-2bc7f80bbb4f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:18.327628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d834e0-8fa2-4cd9-9f76-2bc7f80bbb4f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b1d834e0-8fa2-4cd9-9f76-2bc7f80bbb4f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:30:18.338715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:18.340168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:18.341343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:18.341343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:18.341343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:30:18.341343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:30:18.341343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:30:18.341343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:30:18.341343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:30:18.341343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:30:18.341343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 08:30:18.341343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:18.341343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 641/1267] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:18.346718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe474-cdaf-4006-9663-4eb9f9289992 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:18.452904 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe474-cdaf-4006-9663-4eb9f9289992 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:18.492013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156647e0-2f67-44d3-91f8-1482c134dc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:18.492630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156647e0-2f67-44d3-91f8-1482c134dc50 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:18.492905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156647e0-2f67-44d3-91f8-1482c134dc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:18.493172 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156647e0-2f67-44d3-91f8-1482c134dc50 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:18.493992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156647e0-2f67-44d3-91f8-1482c134dc50 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:18.494323 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156647e0-2f67-44d3-91f8-1482c134dc50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:18.495895 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156647e0-2f67-44d3-91f8-1482c134dc50 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:18.506535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:18.516455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 627/1268] 10.222.0.44 () {56 vars in 1143 bytes} [Sun Sep 11 08:30:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:18.522726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d834e0-8fa2-4cd9-9f76-2bc7f80bbb4f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:18.641944 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d834e0-8fa2-4cd9-9f76-2bc7f80bbb4f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:18.680898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac19535-d406-40c8-bf45-67975fd67066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:18.681549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac19535-d406-40c8-bf45-67975fd67066 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:18.681826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac19535-d406-40c8-bf45-67975fd67066 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:18.682092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac19535-d406-40c8-bf45-67975fd67066 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:18.693992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac19535-d406-40c8-bf45-67975fd67066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:18.695483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac19535-d406-40c8-bf45-67975fd67066 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9ac19535-d406-40c8-bf45-67975fd67066 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:30:18.711133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:18.713449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:30:18.714462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:30:18.714462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:18.714462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:18.714462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 642/1269] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:18.718378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156647e0-2f67-44d3-91f8-1482c134dc50 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:18.821363 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156647e0-2f67-44d3-91f8-1482c134dc50 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:18.861583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee7f19-a924-4ba3-b4ad-032931cfd591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:18.862222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee7f19-a924-4ba3-b4ad-032931cfd591 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:18.862497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee7f19-a924-4ba3-b4ad-032931cfd591 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:18.862771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee7f19-a924-4ba3-b4ad-032931cfd591 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:18.863428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee7f19-a924-4ba3-b4ad-032931cfd591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:18.863814 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee7f19-a924-4ba3-b4ad-032931cfd591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:18.865167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee7f19-a924-4ba3-b4ad-032931cfd591 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:18.878196 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 628/1270] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:18.884034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac19535-d406-40c8-bf45-67975fd67066 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:18.919684 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:19.003342 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac19535-d406-40c8-bf45-67975fd67066 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:19.036432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ca876-7716-4999-b364-a660024c6eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:19.037141 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ca876-7716-4999-b364-a660024c6eae None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:19.037415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ca876-7716-4999-b364-a660024c6eae None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:19.037697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ca876-7716-4999-b364-a660024c6eae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:19.045630 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ca876-7716-4999-b364-a660024c6eae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:19.047636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ca876-7716-4999-b364-a660024c6eae None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-445ca876-7716-4999-b364-a660024c6eae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:30:19.059381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:19.062078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:19.063244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:30:19.063244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:30:19.063244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:30:19.063244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:30:19.063244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:30:19.063244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:30:19.063244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:30:19.063244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:19.063244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 643/1271] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:30:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:19.067447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee7f19-a924-4ba3-b4ad-032931cfd591 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:19.126452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:19.171028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee7f19-a924-4ba3-b4ad-032931cfd591 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:19.205957 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51587e46-555c-4717-b417-5388900b6728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:19.206602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51587e46-555c-4717-b417-5388900b6728 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:19.206879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51587e46-555c-4717-b417-5388900b6728 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:19.207147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51587e46-555c-4717-b417-5388900b6728 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:19.207826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51587e46-555c-4717-b417-5388900b6728 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:19.208129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51587e46-555c-4717-b417-5388900b6728 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:19.209575 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51587e46-555c-4717-b417-5388900b6728 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:19.224809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 629/1272] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:30:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:19.231028 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ca876-7716-4999-b364-a660024c6eae None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:19.334304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:19.342849 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ca876-7716-4999-b364-a660024c6eae None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:19.381650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c427af1-c526-43c8-b15e-75ccdd457bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:19.382338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c427af1-c526-43c8-b15e-75ccdd457bbc None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:19.382613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c427af1-c526-43c8-b15e-75ccdd457bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:19.382885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c427af1-c526-43c8-b15e-75ccdd457bbc None admin] PATH_INFO: `/v3/projects/811863e9c94d44a39647800f306505f0/users/8d03db170fb6460794e32669456bcdaa/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:19.384781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c427af1-c526-43c8-b15e-75ccdd457bbc None admin] RBAC: Authorizing `identity:create_grant(project_id=811863e9c94d44a39647800f306505f0, user_id=8d03db170fb6460794e32669456bcdaa, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:19.386221 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c427af1-c526-43c8-b15e-75ccdd457bbc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:19.501584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 644/1273] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 08:30:19 2022] PUT /identity/v3/projects/811863e9c94d44a39647800f306505f0/users/8d03db170fb6460794e32669456bcdaa/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:30:21.665678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e031ff-ff3c-42d0-bf83-bb01ff529761 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:21.666267 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e031ff-ff3c-42d0-bf83-bb01ff529761 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:21.666267 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e031ff-ff3c-42d0-bf83-bb01ff529761 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:21.667604 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 630/1274] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:30:21.673145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ea332-bb68-4452-8fb7-afd5a1f23bbe None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:21.673559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ea332-bb68-4452-8fb7-afd5a1f23bbe None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:21.673724 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ea332-bb68-4452-8fb7-afd5a1f23bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:21.692992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d21ea332-bb68-4452-8fb7-afd5a1f23bbe None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:30:21.694211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d21ea332-bb68-4452-8fb7-afd5a1f23bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:30:21.770046 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 645/1275] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:30:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:30:22.037673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e031ff-ff3c-42d0-bf83-bb01ff529761 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:22.144347 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e031ff-ff3c-42d0-bf83-bb01ff529761 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:22.163222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:22.182052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432326a1-01e8-4b27-919d-e21a70201965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:22.182869 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432326a1-01e8-4b27-919d-e21a70201965 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:22.188538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432326a1-01e8-4b27-919d-e21a70201965 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:22.188538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432326a1-01e8-4b27-919d-e21a70201965 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:22.195072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432326a1-01e8-4b27-919d-e21a70201965 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-432326a1-01e8-4b27-919d-e21a70201965 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 08:30:22.200090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:22.201238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:22.202090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:22.202090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:22.202090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:22.202090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:22.202090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 08:30:22.202090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 08:30:22.202090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 08:30:22.202090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 08:30:22.202090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 08:30:22.202090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 08:30:22.202090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 08:30:22.202090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:22.202090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 631/1276] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 08:30:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:22.208320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ea332-bb68-4452-8fb7-afd5a1f23bbe None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:22.308404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:22.328168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ea332-bb68-4452-8fb7-afd5a1f23bbe None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:22.371248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c9d57-1bf3-4318-9448-c494dacd4acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:22.371855 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9d57-1bf3-4318-9448-c494dacd4acb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:22.372144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9d57-1bf3-4318-9448-c494dacd4acb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:22.372461 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9d57-1bf3-4318-9448-c494dacd4acb None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:22.373655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c9d57-1bf3-4318-9448-c494dacd4acb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:22.373956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c9d57-1bf3-4318-9448-c494dacd4acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:22.375373 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c9d57-1bf3-4318-9448-c494dacd4acb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:22.384516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 646/1277] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 08:30:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:22.394361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432326a1-01e8-4b27-919d-e21a70201965 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:22.524586 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432326a1-01e8-4b27-919d-e21a70201965 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:22.562317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c5ea1-ca8f-49a5-99d8-0786911beb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:22.562832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c5ea1-ca8f-49a5-99d8-0786911beb24 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:22.563134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c5ea1-ca8f-49a5-99d8-0786911beb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:22.563535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c5ea1-ca8f-49a5-99d8-0786911beb24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:22.576412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c5ea1-ca8f-49a5-99d8-0786911beb24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:22.577968 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c5ea1-ca8f-49a5-99d8-0786911beb24 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f17c5ea1-ca8f-49a5-99d8-0786911beb24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:30:22.594063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:22.597581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:30:22.598564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:30:22.598564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:22.598564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:22.598564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 632/1278] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:22.601415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c9d57-1bf3-4318-9448-c494dacd4acb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:22.736336 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c9d57-1bf3-4318-9448-c494dacd4acb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:22.771294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672542ea-6f82-441f-bf65-521dfad7e3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:22.771902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672542ea-6f82-441f-bf65-521dfad7e3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:22.772178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672542ea-6f82-441f-bf65-521dfad7e3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:22.772444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672542ea-6f82-441f-bf65-521dfad7e3d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:22.773177 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672542ea-6f82-441f-bf65-521dfad7e3d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:22.773462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672542ea-6f82-441f-bf65-521dfad7e3d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:22.774886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672542ea-6f82-441f-bf65-521dfad7e3d8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:22.791323 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 647/1279] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:22.797319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c5ea1-ca8f-49a5-99d8-0786911beb24 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:22.840271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:22.909289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c5ea1-ca8f-49a5-99d8-0786911beb24 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:22.944850 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce693c-71bb-46e0-9288-c6cf98087502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:22.945551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce693c-71bb-46e0-9288-c6cf98087502 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:22.949421 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce693c-71bb-46e0-9288-c6cf98087502 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:22.949421 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce693c-71bb-46e0-9288-c6cf98087502 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:22.957118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce693c-71bb-46e0-9288-c6cf98087502 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:22.958707 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce693c-71bb-46e0-9288-c6cf98087502 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d9ce693c-71bb-46e0-9288-c6cf98087502 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 08:30:22.972829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:22.974028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:22.975080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:22.975080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:22.975080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 08:30:22.975080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 08:30:22.975080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 08:30:22.975080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 08:30:22.975080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 08:30:22.975080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 08:30:22.975080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 08:30:22.975080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:22.975080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 633/1280] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:22.980162 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672542ea-6f82-441f-bf65-521dfad7e3d8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:22.984440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:23.081390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672542ea-6f82-441f-bf65-521dfad7e3d8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:23.124386 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d924cf-7a05-4790-b09b-fd28f666b876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:23.125033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d924cf-7a05-4790-b09b-fd28f666b876 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:23.125307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d924cf-7a05-4790-b09b-fd28f666b876 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:23.125819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d924cf-7a05-4790-b09b-fd28f666b876 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:23.127362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d924cf-7a05-4790-b09b-fd28f666b876 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:23.127742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d924cf-7a05-4790-b09b-fd28f666b876 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:23.129203 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d924cf-7a05-4790-b09b-fd28f666b876 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:23.154301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 648/1281] 10.222.0.44 () {56 vars in 1143 bytes} [Sun Sep 11 08:30:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:23.155630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce693c-71bb-46e0-9288-c6cf98087502 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:23.269523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:23.283012 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce693c-71bb-46e0-9288-c6cf98087502 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:23.327620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf8b76-080d-41be-bea1-da64a8221558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:23.328834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf8b76-080d-41be-bea1-da64a8221558 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:23.328834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf8b76-080d-41be-bea1-da64a8221558 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:23.328834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf8b76-080d-41be-bea1-da64a8221558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:23.404783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf8b76-080d-41be-bea1-da64a8221558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:23.406166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf8b76-080d-41be-bea1-da64a8221558 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:23.513690 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:23.523978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 634/1282] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 08:30:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 08:30:23.530443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d924cf-7a05-4790-b09b-fd28f666b876 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:23.607300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:23.650026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d924cf-7a05-4790-b09b-fd28f666b876 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:23.686311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c130b-ef49-4189-819f-09e8465f75b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:23.686311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c130b-ef49-4189-819f-09e8465f75b0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:23.686311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c130b-ef49-4189-819f-09e8465f75b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:23.686311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c130b-ef49-4189-819f-09e8465f75b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:23.696856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c130b-ef49-4189-819f-09e8465f75b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:23.698338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c130b-ef49-4189-819f-09e8465f75b0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-de3c130b-ef49-4189-819f-09e8465f75b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 08:30:23.713118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:23.714325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 08:30:23.715466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 08:30:23.715466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 08:30:23.715466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:23.715466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 649/1283] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:30:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:30:23.721963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf8b76-080d-41be-bea1-da64a8221558 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:23.827581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf8b76-080d-41be-bea1-da64a8221558 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:23.864506 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5b62d-cf3f-4bb8-9b78-101978075759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:23.865116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5b62d-cf3f-4bb8-9b78-101978075759 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:23.865486 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5b62d-cf3f-4bb8-9b78-101978075759 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:23.865909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5b62d-cf3f-4bb8-9b78-101978075759 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:23.866582 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5b62d-cf3f-4bb8-9b78-101978075759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:23.869785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5b62d-cf3f-4bb8-9b78-101978075759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:23.869785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5b62d-cf3f-4bb8-9b78-101978075759 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:23.887123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 635/1284] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 08:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:30:23.889310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c130b-ef49-4189-819f-09e8465f75b0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:23.991699 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c130b-ef49-4189-819f-09e8465f75b0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:24.005653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:24.025742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90807f8-b0d6-4f6d-bb19-4c0950dc51b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:24.026403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90807f8-b0d6-4f6d-bb19-4c0950dc51b6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:24.026679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90807f8-b0d6-4f6d-bb19-4c0950dc51b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:24.027032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90807f8-b0d6-4f6d-bb19-4c0950dc51b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:24.034460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90807f8-b0d6-4f6d-bb19-4c0950dc51b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:24.036024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90807f8-b0d6-4f6d-bb19-4c0950dc51b6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e90807f8-b0d6-4f6d-bb19-4c0950dc51b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 08:30:24.048536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:24.053038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:24.054007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 08:30:24.054007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 08:30:24.054007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 08:30:24.054007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 08:30:24.054007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 08:30:24.054007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 08:30:24.054007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 08:30:24.054007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Sep 11 08:30:24.054007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 650/1285] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 08:30:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 08:30:24.055757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5b62d-cf3f-4bb8-9b78-101978075759 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:24.112805 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:24.161689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5b62d-cf3f-4bb8-9b78-101978075759 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:24.199920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fff9d-4b7a-4627-b04c-2f68af0996a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:24.200545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fff9d-4b7a-4627-b04c-2f68af0996a9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:24.200817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fff9d-4b7a-4627-b04c-2f68af0996a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:24.201071 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fff9d-4b7a-4627-b04c-2f68af0996a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:24.201761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fff9d-4b7a-4627-b04c-2f68af0996a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:24.202062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fff9d-4b7a-4627-b04c-2f68af0996a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:24.203537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fff9d-4b7a-4627-b04c-2f68af0996a9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:24.215633 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 636/1286] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 08:30:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:24.222606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90807f8-b0d6-4f6d-bb19-4c0950dc51b6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:24.335502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90807f8-b0d6-4f6d-bb19-4c0950dc51b6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:24.369845 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab069795-f89a-4090-bbfa-666071ee77a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:24.370467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab069795-f89a-4090-bbfa-666071ee77a8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:24.370736 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab069795-f89a-4090-bbfa-666071ee77a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:24.370999 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab069795-f89a-4090-bbfa-666071ee77a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:24.371709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab069795-f89a-4090-bbfa-666071ee77a8 None admin] RBAC: Adding query filter params (role.id=020bc02dca71485bab2312442adaa179, scope.project.id=811863e9c94d44a39647800f306505f0, user.id=8d03db170fb6460794e32669456bcdaa) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:24.372011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab069795-f89a-4090-bbfa-666071ee77a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:24.373522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab069795-f89a-4090-bbfa-666071ee77a8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:24.379425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 651/1287] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 08:30:24 2022] GET /identity/v3/role_assignments?user.id=8d03db170fb6460794e32669456bcdaa&scope.project.id=811863e9c94d44a39647800f306505f0&role.id=020bc02dca71485bab2312442adaa179 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:26.584739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e215-ad00-4611-a024-b31168e9c130 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:26.585316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e215-ad00-4611-a024-b31168e9c130 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:26.585316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e215-ad00-4611-a024-b31168e9c130 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:26.586718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 637/1288] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:30:26.595017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c482a0-fa62-4087-8720-3aac79f41b7b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:26.595501 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c482a0-fa62-4087-8720-3aac79f41b7b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:26.595594 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c482a0-fa62-4087-8720-3aac79f41b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:26.614012 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c0c482a0-fa62-4087-8720-3aac79f41b7b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:30:26.615219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c0c482a0-fa62-4087-8720-3aac79f41b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:30:26.647130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:26.684323 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 652/1289] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:30:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:30:26.949826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9e215-ad00-4611-a024-b31168e9c130 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:27.054720 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9e215-ad00-4611-a024-b31168e9c130 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:27.065980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:27.088771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a028f2-f85e-4c77-b527-1e3e97c5b8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:27.089396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a028f2-f85e-4c77-b527-1e3e97c5b8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:27.089667 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a028f2-f85e-4c77-b527-1e3e97c5b8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:27.089931 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a028f2-f85e-4c77-b527-1e3e97c5b8d6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:27.090542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a028f2-f85e-4c77-b527-1e3e97c5b8d6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:27.092471 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a028f2-f85e-4c77-b527-1e3e97c5b8d6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c4a028f2-f85e-4c77-b527-1e3e97c5b8d6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:27.101533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:27.102777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:27.103653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 638/1290] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 08:30:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:27.109274 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c482a0-fa62-4087-8720-3aac79f41b7b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:27.211872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c482a0-fa62-4087-8720-3aac79f41b7b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:27.247049 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1a161-5f70-4a1e-bec8-a8c75335e931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:27.247653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1a161-5f70-4a1e-bec8-a8c75335e931 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:27.247929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1a161-5f70-4a1e-bec8-a8c75335e931 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:27.248430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1a161-5f70-4a1e-bec8-a8c75335e931 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:27.248912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1a161-5f70-4a1e-bec8-a8c75335e931 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:27.249207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1a161-5f70-4a1e-bec8-a8c75335e931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:27.250599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1a161-5f70-4a1e-bec8-a8c75335e931 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:27.256297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 653/1291] 10.222.0.44 () {56 vars in 1116 bytes} [Sun Sep 11 08:30:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:27.264819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a028f2-f85e-4c77-b527-1e3e97c5b8d6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:27.364557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a028f2-f85e-4c77-b527-1e3e97c5b8d6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:27.399490 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410b882-f04f-4f59-9568-d938ad43b1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:27.400202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410b882-f04f-4f59-9568-d938ad43b1bf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:27.400446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410b882-f04f-4f59-9568-d938ad43b1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:27.400767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410b882-f04f-4f59-9568-d938ad43b1bf None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:27.401386 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410b882-f04f-4f59-9568-d938ad43b1bf None admin] RBAC: Adding query filter params (type=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:27.401681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410b882-f04f-4f59-9568-d938ad43b1bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:27.403072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410b882-f04f-4f59-9568-d938ad43b1bf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:27.408351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 639/1292] 10.222.0.44 () {56 vars in 1116 bytes} [Sun Sep 11 08:30:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:29.686401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf3175-e2b6-43f4-83a0-58fe63ec5f0d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:29.686998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf3175-e2b6-43f4-83a0-58fe63ec5f0d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:29.686998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf3175-e2b6-43f4-83a0-58fe63ec5f0d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:29.688376 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 654/1293] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:30:29.693829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0970eca-49f0-4558-9c50-3a18c8754d29 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:29.694315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0970eca-49f0-4558-9c50-3a18c8754d29 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:29.694405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0970eca-49f0-4558-9c50-3a18c8754d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:29.713413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a0970eca-49f0-4558-9c50-3a18c8754d29 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:30:29.714638 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a0970eca-49f0-4558-9c50-3a18c8754d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:30:29.748808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:29.787327 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 640/1294] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:30:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:30:30.060773 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf3175-e2b6-43f4-83a0-58fe63ec5f0d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:30.095545 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:30.172740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf3175-e2b6-43f4-83a0-58fe63ec5f0d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:30.214548 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e4aed-e0b2-4a82-a55a-ee1b85376cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:30.215184 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e4aed-e0b2-4a82-a55a-ee1b85376cf2 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:30.215459 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e4aed-e0b2-4a82-a55a-ee1b85376cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:30.215727 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e4aed-e0b2-4a82-a55a-ee1b85376cf2 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:30.216450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e4aed-e0b2-4a82-a55a-ee1b85376cf2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:30.217882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e4aed-e0b2-4a82-a55a-ee1b85376cf2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:30.303431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 655/1295] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 08:30:30 2022] POST /identity/v3/services => generated 246 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:30:32.534290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963fc09-5a91-491b-b121-a8a0f4e6ce70 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:32.534926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963fc09-5a91-491b-b121-a8a0f4e6ce70 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:32.534926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963fc09-5a91-491b-b121-a8a0f4e6ce70 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:32.536175 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 641/1296] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:30:32.542115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a0eb5-0ce4-43ad-a2d8-96a99df7099f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:32.542678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a0eb5-0ce4-43ad-a2d8-96a99df7099f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:32.542678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a0eb5-0ce4-43ad-a2d8-96a99df7099f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:32.560742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bc3a0eb5-0ce4-43ad-a2d8-96a99df7099f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:30:32.561976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bc3a0eb5-0ce4-43ad-a2d8-96a99df7099f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:30:32.640260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 656/1297] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:30:32 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:30:32.912760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963fc09-5a91-491b-b121-a8a0f4e6ce70 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:33.017602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963fc09-5a91-491b-b121-a8a0f4e6ce70 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:33.053616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ad973-11c1-4e25-871a-7b66e1724742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:33.054267 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ad973-11c1-4e25-871a-7b66e1724742 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:33.054548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ad973-11c1-4e25-871a-7b66e1724742 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:33.054819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ad973-11c1-4e25-871a-7b66e1724742 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:33.055400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ad973-11c1-4e25-871a-7b66e1724742 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:33.056799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ad973-11c1-4e25-871a-7b66e1724742 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-619ad973-11c1-4e25-871a-7b66e1724742 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:33.066374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:33.067695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:33.068836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 642/1298] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 08:30:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:33.073728 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a0eb5-0ce4-43ad-a2d8-96a99df7099f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:33.092358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:33.171756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a0eb5-0ce4-43ad-a2d8-96a99df7099f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:33.206375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d7376-9a7c-4e5a-bd53-b7000ea35c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:33.206972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d7376-9a7c-4e5a-bd53-b7000ea35c36 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:33.207245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d7376-9a7c-4e5a-bd53-b7000ea35c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:33.207512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d7376-9a7c-4e5a-bd53-b7000ea35c36 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:33.208215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d7376-9a7c-4e5a-bd53-b7000ea35c36 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:33.208511 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d7376-9a7c-4e5a-bd53-b7000ea35c36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:33.209856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d7376-9a7c-4e5a-bd53-b7000ea35c36 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:33.215127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 657/1299] 10.222.0.44 () {56 vars in 1116 bytes} [Sun Sep 11 08:30:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:33.221338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ad973-11c1-4e25-871a-7b66e1724742 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:33.256715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:33.330561 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ad973-11c1-4e25-871a-7b66e1724742 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:33.362967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073614e8-ac1f-4d10-92fa-8169ad42192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:33.363573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073614e8-ac1f-4d10-92fa-8169ad42192e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:33.363852 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073614e8-ac1f-4d10-92fa-8169ad42192e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:33.364117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073614e8-ac1f-4d10-92fa-8169ad42192e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:33.364811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073614e8-ac1f-4d10-92fa-8169ad42192e None admin] RBAC: Adding query filter params (interface=public, service_id=4b7b877310e44743b690c7d02f31ea1c, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:33.365109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073614e8-ac1f-4d10-92fa-8169ad42192e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:33.366495 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073614e8-ac1f-4d10-92fa-8169ad42192e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:33.371864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 643/1300] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 08:30:33 2022] GET /identity/v3/endpoints?service_id=4b7b877310e44743b690c7d02f31ea1c&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:33.378393 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d7376-9a7c-4e5a-bd53-b7000ea35c36 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:33.460966 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:33.479913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d7376-9a7c-4e5a-bd53-b7000ea35c36 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:33.514885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d9332-c606-44ba-8e10-343f95518d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:33.515503 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9332-c606-44ba-8e10-343f95518d08 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:33.515968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9332-c606-44ba-8e10-343f95518d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:33.516627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9332-c606-44ba-8e10-343f95518d08 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:33.517229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d9332-c606-44ba-8e10-343f95518d08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:33.518626 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d9332-c606-44ba-8e10-343f95518d08 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:33.525574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 658/1301] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 08:30:33 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:30:35.690641 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fc210-8839-42d1-ba3b-b8b05863b1cb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:35.691304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fc210-8839-42d1-ba3b-b8b05863b1cb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:35.691304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fc210-8839-42d1-ba3b-b8b05863b1cb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:35.692529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 644/1302] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:30:35.698788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a303ab-538b-446d-b63a-082a5e93f97f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:35.699600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a303ab-538b-446d-b63a-082a5e93f97f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:35.699600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a303ab-538b-446d-b63a-082a5e93f97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:35.720769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a1a303ab-538b-446d-b63a-082a5e93f97f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:30:35.721991 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a1a303ab-538b-446d-b63a-082a5e93f97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:30:35.799778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 659/1303] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:30:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 08:30:36.067196 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fc210-8839-42d1-ba3b-b8b05863b1cb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:36.164744 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:36.172012 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fc210-8839-42d1-ba3b-b8b05863b1cb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:36.207076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33dbe0b-6763-4a4f-b69b-8ab52730988d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:36.207700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33dbe0b-6763-4a4f-b69b-8ab52730988d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:36.207969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33dbe0b-6763-4a4f-b69b-8ab52730988d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:36.208231 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33dbe0b-6763-4a4f-b69b-8ab52730988d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:36.208804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33dbe0b-6763-4a4f-b69b-8ab52730988d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:36.210188 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33dbe0b-6763-4a4f-b69b-8ab52730988d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d33dbe0b-6763-4a4f-b69b-8ab52730988d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 08:30:36.220412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 08:30:36.222444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Sep 11 08:30:36.228540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 645/1304] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 08:30:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 08:30:36.230018 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a303ab-538b-446d-b63a-082a5e93f97f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:36.309664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:36.333275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a303ab-538b-446d-b63a-082a5e93f97f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:36.368601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb6581-e720-4172-9f75-67c20f0bccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:36.369210 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb6581-e720-4172-9f75-67c20f0bccee None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:36.369485 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb6581-e720-4172-9f75-67c20f0bccee None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:36.369759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb6581-e720-4172-9f75-67c20f0bccee None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:36.370417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb6581-e720-4172-9f75-67c20f0bccee None admin] RBAC: Adding query filter params (name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:30:36.370712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb6581-e720-4172-9f75-67c20f0bccee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:36.372636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb6581-e720-4172-9f75-67c20f0bccee None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:36.377988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 660/1305] 10.222.0.44 () {56 vars in 1116 bytes} [Sun Sep 11 08:30:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:30:36.383958 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33dbe0b-6763-4a4f-b69b-8ab52730988d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:30:36.486867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33dbe0b-6763-4a4f-b69b-8ab52730988d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:30:36.521556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd7a58-f37d-433a-9daa-e5282df90d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:30:36.522204 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd7a58-f37d-433a-9daa-e5282df90d9a None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:30:36.522479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd7a58-f37d-433a-9daa-e5282df90d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:30:36.522741 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd7a58-f37d-433a-9daa-e5282df90d9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:30:36.523396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd7a58-f37d-433a-9daa-e5282df90d9a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:30:36.524755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd7a58-f37d-433a-9daa-e5282df90d9a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:30:36.630522 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 646/1306] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:30:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:30:46.397698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=30 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:30:46.623240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:16.396542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01f8f8-1f8e-4f66-901e-569cdde64a51 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:16.397185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01f8f8-1f8e-4f66-901e-569cdde64a51 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:16.397185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01f8f8-1f8e-4f66-901e-569cdde64a51 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:16.404728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b1c03-b383-45e3-8455-d94372bdc786 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:16.405290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b1c03-b383-45e3-8455-d94372bdc786 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:16.405290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b1c03-b383-45e3-8455-d94372bdc786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:16.410692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 661/1307] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:38:16 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:38:16.507930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5c8b1c03-b383-45e3-8455-d94372bdc786 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:16.509180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5c8b1c03-b383-45e3-8455-d94372bdc786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:16.662720 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 647/1308] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:38:17.020140 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d8c4c-0d53-451a-b562-900eebacd59a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:17.020448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d8c4c-0d53-451a-b562-900eebacd59a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:17.020690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d8c4c-0d53-451a-b562-900eebacd59a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:17.022442 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 662/1309] 10.222.0.44 () {54 vars in 965 bytes} [Sun Sep 11 08:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:38:17.030004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61922c6d-ae70-4e6c-afc0-fe3580f722e0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:17.031006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61922c6d-ae70-4e6c-afc0-fe3580f722e0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:17.031785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61922c6d-ae70-4e6c-afc0-fe3580f722e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:17.139042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-61922c6d-ae70-4e6c-afc0-fe3580f722e0 None None] MFA Rules not processed for user `92867e15dcf649adad78bf7a29a996f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:17.142055 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-61922c6d-ae70-4e6c-afc0-fe3580f722e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:17.166017 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:17.226596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 648/1310] 10.222.0.44 () {58 vars in 1062 bytes} [Sun Sep 11 08:38:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:38:17.235678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d8c4c-0d53-451a-b562-900eebacd59a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:38:17.237423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-de2d8c4c-0d53-451a-b562-900eebacd59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:17.352402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d8c4c-0d53-451a-b562-900eebacd59a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:38:17.432663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d72a1d-9387-4531-9551-f786b53d135b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:38:17.433308 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d72a1d-9387-4531-9551-f786b53d135b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:17.433515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d72a1d-9387-4531-9551-f786b53d135b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:17.433816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d72a1d-9387-4531-9551-f786b53d135b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:17.435670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-86d72a1d-9387-4531-9551-f786b53d135b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:17.493371 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:17.557846 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d72a1d-9387-4531-9551-f786b53d135b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:38:17.559348 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d72a1d-9387-4531-9551-f786b53d135b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:38:17.581357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 663/1311] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:38:20.058587 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612e0b8-fe48-4cb1-9ad0-5f36eb0069c4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:20.059129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612e0b8-fe48-4cb1-9ad0-5f36eb0069c4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:20.059129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612e0b8-fe48-4cb1-9ad0-5f36eb0069c4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:20.060457 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 649/1312] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:38:20.066516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fc2de-3078-421f-b130-11a085a15757 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:20.067020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fc2de-3078-421f-b130-11a085a15757 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:20.067118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fc2de-3078-421f-b130-11a085a15757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:20.142251 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-715fc2de-3078-421f-b130-11a085a15757 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:20.143420 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-715fc2de-3078-421f-b130-11a085a15757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:20.239568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 664/1313] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:38:20.533283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612e0b8-fe48-4cb1-9ad0-5f36eb0069c4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:38:20.534909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4612e0b8-fe48-4cb1-9ad0-5f36eb0069c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:20.641361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612e0b8-fe48-4cb1-9ad0-5f36eb0069c4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:38:20.713924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:20.716273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed101509-f58d-4a72-949a-472972c5f3c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:38:20.716889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed101509-f58d-4a72-949a-472972c5f3c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:20.717164 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed101509-f58d-4a72-949a-472972c5f3c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:20.717483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed101509-f58d-4a72-949a-472972c5f3c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:20.719069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ed101509-f58d-4a72-949a-472972c5f3c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:20.832457 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed101509-f58d-4a72-949a-472972c5f3c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:38:20.833877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed101509-f58d-4a72-949a-472972c5f3c1 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:38:20.853151 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 650/1314] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:38:23.567876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54ced9-0a31-4cbd-b7bc-8d718d422a42 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:23.568448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54ced9-0a31-4cbd-b7bc-8d718d422a42 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:23.568448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54ced9-0a31-4cbd-b7bc-8d718d422a42 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:23.569895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 665/1315] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:38:23.575704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f530a-f978-4f4d-b744-dcb9519f86a5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:23.576187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f530a-f978-4f4d-b744-dcb9519f86a5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:23.576302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f530a-f978-4f4d-b744-dcb9519f86a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:23.644759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2c7f530a-f978-4f4d-b744-dcb9519f86a5 None None] MFA Rules not processed for user `107dc688e53144dea3e37d8737d74742`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:23.645932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c7f530a-f978-4f4d-b744-dcb9519f86a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:23.699283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:23.751200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 651/1316] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:38:23 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:38:24.035063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54ced9-0a31-4cbd-b7bc-8d718d422a42 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:38:24.036627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6e54ced9-0a31-4cbd-b7bc-8d718d422a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:24.048106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:24.140761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54ced9-0a31-4cbd-b7bc-8d718d422a42 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:38:24.214463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b720b6-3de8-4ab0-9d64-a0b47622db0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:38:24.215062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b720b6-3de8-4ab0-9d64-a0b47622db0a service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:24.215337 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b720b6-3de8-4ab0-9d64-a0b47622db0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:24.215604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b720b6-3de8-4ab0-9d64-a0b47622db0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:24.217230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c5b720b6-3de8-4ab0-9d64-a0b47622db0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:24.280687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:24.326853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b720b6-3de8-4ab0-9d64-a0b47622db0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:38:24.328288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b720b6-3de8-4ab0-9d64-a0b47622db0a service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:38:24.347496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 666/1317] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:38:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:38:27.425477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943d51d-f8a4-452a-a584-770367f619a6 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:27.426022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943d51d-f8a4-452a-a584-770367f619a6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:27.426022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943d51d-f8a4-452a-a584-770367f619a6 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:27.427413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 652/1318] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:38:27.432841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46c7b7-15ae-4913-92a8-6e43d10a9727 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:27.433297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46c7b7-15ae-4913-92a8-6e43d10a9727 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:27.433396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46c7b7-15ae-4913-92a8-6e43d10a9727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:27.499840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0e46c7b7-15ae-4913-92a8-6e43d10a9727 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:27.501044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0e46c7b7-15ae-4913-92a8-6e43d10a9727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:27.597750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 667/1319] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:38:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:38:27.908253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340789e-97dd-402d-a10e-dd036965354c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:27.908532 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340789e-97dd-402d-a10e-dd036965354c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:27.908808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340789e-97dd-402d-a10e-dd036965354c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:27.910275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 653/1320] 10.222.0.44 () {54 vars in 965 bytes} [Sun Sep 11 08:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:38:27.916635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec96f80-6674-4d3a-b13f-103975b8fbc1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:27.916913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec96f80-6674-4d3a-b13f-103975b8fbc1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:27.917211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec96f80-6674-4d3a-b13f-103975b8fbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:27.939820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:28.001993 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-aec96f80-6674-4d3a-b13f-103975b8fbc1 None None] MFA Rules not processed for user `92867e15dcf649adad78bf7a29a996f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:28.003272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aec96f80-6674-4d3a-b13f-103975b8fbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:28.079200 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 668/1321] 10.222.0.44 () {58 vars in 1062 bytes} [Sun Sep 11 08:38:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:38:28.088361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340789e-97dd-402d-a10e-dd036965354c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:38:28.089984 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7340789e-97dd-402d-a10e-dd036965354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:28.197045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340789e-97dd-402d-a10e-dd036965354c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:38:28.275905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21227146-0f21-4944-b5d3-df34792fb79f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:38:28.276516 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21227146-0f21-4944-b5d3-df34792fb79f service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:28.276790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21227146-0f21-4944-b5d3-df34792fb79f service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:28.277055 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21227146-0f21-4944-b5d3-df34792fb79f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:28.278785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-21227146-0f21-4944-b5d3-df34792fb79f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:28.287045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:28.410646 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21227146-0f21-4944-b5d3-df34792fb79f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:38:28.414019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21227146-0f21-4944-b5d3-df34792fb79f service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:38:28.436740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 654/1322] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:38:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:38:32.075644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd7e51-943e-4f48-86a7-7f0932b6dc21 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:32.076213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd7e51-943e-4f48-86a7-7f0932b6dc21 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:32.076307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd7e51-943e-4f48-86a7-7f0932b6dc21 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:32.077727 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 669/1323] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:38:32.083434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7cbd9-2ac7-480d-9453-2393b9add03f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:32.084479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7cbd9-2ac7-480d-9453-2393b9add03f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:32.084604 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7cbd9-2ac7-480d-9453-2393b9add03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:32.171625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-def7cbd9-2ac7-480d-9453-2393b9add03f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:32.171625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-def7cbd9-2ac7-480d-9453-2393b9add03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:32.236550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:32.288010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 655/1324] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:38:32.573356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd7e51-943e-4f48-86a7-7f0932b6dc21 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:38:32.574921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-08dd7e51-943e-4f48-86a7-7f0932b6dc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:32.696935 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd7e51-943e-4f48-86a7-7f0932b6dc21 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:38:32.718508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:32.786862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2dd99-6fc7-4049-ada1-5d4bdc386af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:38:32.787592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2dd99-6fc7-4049-ada1-5d4bdc386af1 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:32.787930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2dd99-6fc7-4049-ada1-5d4bdc386af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:32.788281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2dd99-6fc7-4049-ada1-5d4bdc386af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:32.790020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-01a2dd99-6fc7-4049-ada1-5d4bdc386af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:32.908266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2dd99-6fc7-4049-ada1-5d4bdc386af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:38:32.909697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2dd99-6fc7-4049-ada1-5d4bdc386af1 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:38:32.928460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 670/1325] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:38:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:38:35.253468 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca000b36-d7bf-4fdb-9e62-7b8d347fb686 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:35.254008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca000b36-d7bf-4fdb-9e62-7b8d347fb686 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:35.254008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca000b36-d7bf-4fdb-9e62-7b8d347fb686 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:35.255423 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 656/1326] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:38:35.261806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6fdb5-93f2-4e7b-9106-7067f92e9ac1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:35.262348 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6fdb5-93f2-4e7b-9106-7067f92e9ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:35.262706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6fdb5-93f2-4e7b-9106-7067f92e9ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:35.327179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:35.346629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e7d6fdb5-93f2-4e7b-9106-7067f92e9ac1 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:35.347846 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e7d6fdb5-93f2-4e7b-9106-7067f92e9ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:35.471441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 671/1327] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:38:35.759224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca000b36-d7bf-4fdb-9e62-7b8d347fb686 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:38:35.760803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ca000b36-d7bf-4fdb-9e62-7b8d347fb686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:35.888179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca000b36-d7bf-4fdb-9e62-7b8d347fb686 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:38:35.980097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1135239-6207-4000-b328-4a83dfd7b9ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:38:35.980721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135239-6207-4000-b328-4a83dfd7b9ee service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:35.980994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135239-6207-4000-b328-4a83dfd7b9ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:35.981259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135239-6207-4000-b328-4a83dfd7b9ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:35.982994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e1135239-6207-4000-b328-4a83dfd7b9ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:36.006386 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:36.109688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1135239-6207-4000-b328-4a83dfd7b9ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:38:36.111240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1135239-6207-4000-b328-4a83dfd7b9ee service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:38:36.133882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 657/1328] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:38:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:38:38.370062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b0ef0-52f4-4e25-9a6d-f61d2f78a7f9 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:38.370630 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b0ef0-52f4-4e25-9a6d-f61d2f78a7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:38.370630 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b0ef0-52f4-4e25-9a6d-f61d2f78a7f9 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:38.371968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 672/1329] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:38:38.377229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a3114-e93d-49bc-84f5-ba8ff7eff150 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:38.377656 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a3114-e93d-49bc-84f5-ba8ff7eff150 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:38.377770 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a3114-e93d-49bc-84f5-ba8ff7eff150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:38.447267 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e16a3114-e93d-49bc-84f5-ba8ff7eff150 None None] MFA Rules not processed for user `107dc688e53144dea3e37d8737d74742`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:38.448514 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e16a3114-e93d-49bc-84f5-ba8ff7eff150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:38.519833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:38.558815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 658/1330] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:38:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:38:38.841891 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b0ef0-52f4-4e25-9a6d-f61d2f78a7f9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:38:38.843579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-178b0ef0-52f4-4e25-9a6d-f61d2f78a7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:38.968165 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b0ef0-52f4-4e25-9a6d-f61d2f78a7f9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:38:38.991893 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:39.051135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8653320-cfde-4e56-a56f-4b6d175d971e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:38:39.051747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8653320-cfde-4e56-a56f-4b6d175d971e service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:39.052018 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8653320-cfde-4e56-a56f-4b6d175d971e service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:39.052284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8653320-cfde-4e56-a56f-4b6d175d971e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:39.054427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a8653320-cfde-4e56-a56f-4b6d175d971e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:39.164984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8653320-cfde-4e56-a56f-4b6d175d971e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:38:39.166447 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8653320-cfde-4e56-a56f-4b6d175d971e service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:38:39.185202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 673/1331] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:38:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:38:41.825784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383df22-6614-493a-97a7-c622029e6b10 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:41.826376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383df22-6614-493a-97a7-c622029e6b10 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:41.826376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383df22-6614-493a-97a7-c622029e6b10 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:41.827678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 659/1332] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:38:41.833030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197a0a1-feb1-41e4-a961-36988251e519 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:41.833415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197a0a1-feb1-41e4-a961-36988251e519 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:41.833588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197a0a1-feb1-41e4-a961-36988251e519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:41.900390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:41.903239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9197a0a1-feb1-41e4-a961-36988251e519 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:41.904461 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9197a0a1-feb1-41e4-a961-36988251e519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:42.008095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 674/1333] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:38:42.289226 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383df22-6614-493a-97a7-c622029e6b10 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:38:42.290834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c383df22-6614-493a-97a7-c622029e6b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:42.419771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383df22-6614-493a-97a7-c622029e6b10 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:38:42.510755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5a7f3-e55b-44e7-a0e6-2a92b31ed5de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:38:42.511377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5a7f3-e55b-44e7-a0e6-2a92b31ed5de service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:42.511653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5a7f3-e55b-44e7-a0e6-2a92b31ed5de service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:42.511925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5a7f3-e55b-44e7-a0e6-2a92b31ed5de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:42.513702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-28c5a7f3-e55b-44e7-a0e6-2a92b31ed5de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:42.545172 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:42.634313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5a7f3-e55b-44e7-a0e6-2a92b31ed5de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:38:42.635798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5a7f3-e55b-44e7-a0e6-2a92b31ed5de service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:38:42.655949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 660/1334] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:38:44.867555 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2c411-7a6a-48cb-8b5c-2b2fd6ee7bb2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:44.868205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2c411-7a6a-48cb-8b5c-2b2fd6ee7bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:44.868205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2c411-7a6a-48cb-8b5c-2b2fd6ee7bb2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:44.869444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 675/1335] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:38:44.875137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba15aa-458a-437a-a011-dadb42467278 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:44.875532 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba15aa-458a-437a-a011-dadb42467278 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:44.875679 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba15aa-458a-437a-a011-dadb42467278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:44.939862 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-19ba15aa-458a-437a-a011-dadb42467278 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:44.941067 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-19ba15aa-458a-437a-a011-dadb42467278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:45.015895 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:45.038291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 661/1336] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:38:45.323695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2c411-7a6a-48cb-8b5c-2b2fd6ee7bb2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:38:45.325234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aaf2c411-7a6a-48cb-8b5c-2b2fd6ee7bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:45.429666 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2c411-7a6a-48cb-8b5c-2b2fd6ee7bb2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:38:45.459106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:45.506362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adef4de-2ac8-4e0f-9aaa-5a0e3b09269b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:38:45.506961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adef4de-2ac8-4e0f-9aaa-5a0e3b09269b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:45.507233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adef4de-2ac8-4e0f-9aaa-5a0e3b09269b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:45.507496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adef4de-2ac8-4e0f-9aaa-5a0e3b09269b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:45.509082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8adef4de-2ac8-4e0f-9aaa-5a0e3b09269b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:45.619706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adef4de-2ac8-4e0f-9aaa-5a0e3b09269b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:38:45.621056 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adef4de-2ac8-4e0f-9aaa-5a0e3b09269b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:38:45.639945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 676/1337] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:38:48.454411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d59e46-7c57-4f73-a132-409d8aebe9da None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:48.454961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d59e46-7c57-4f73-a132-409d8aebe9da None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:48.454961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d59e46-7c57-4f73-a132-409d8aebe9da None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:48.456290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 662/1338] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:38:48.462440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d6b2d-6e10-4932-adee-48affed59f0c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:48.462905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d6b2d-6e10-4932-adee-48affed59f0c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:48.463016 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d6b2d-6e10-4932-adee-48affed59f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:48.534644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-855d6b2d-6e10-4932-adee-48affed59f0c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:48.536526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-855d6b2d-6e10-4932-adee-48affed59f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:48.544464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:48.638756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 677/1339] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:38:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:38:48.927562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d59e46-7c57-4f73-a132-409d8aebe9da None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:38:48.929149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-09d59e46-7c57-4f73-a132-409d8aebe9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:49.035177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d59e46-7c57-4f73-a132-409d8aebe9da None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:38:49.113386 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b40c05-3c3c-403a-9014-a43275bc4b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:38:49.113999 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b40c05-3c3c-403a-9014-a43275bc4b4e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:49.114315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b40c05-3c3c-403a-9014-a43275bc4b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:49.114588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b40c05-3c3c-403a-9014-a43275bc4b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:49.116209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4b40c05-3c3c-403a-9014-a43275bc4b4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:49.153898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:49.231849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b40c05-3c3c-403a-9014-a43275bc4b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:38:49.233292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b40c05-3c3c-403a-9014-a43275bc4b4e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:38:49.254953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 663/1340] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:38:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:38:53.956238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59e8de-a1f7-4fb7-80f1-148a718c9793 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:53.956970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59e8de-a1f7-4fb7-80f1-148a718c9793 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:53.956970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59e8de-a1f7-4fb7-80f1-148a718c9793 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:53.958227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 678/1341] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:38:53.963904 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547f6a6-e018-45c8-aa4b-df2a8a41d1b8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:53.964391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547f6a6-e018-45c8-aa4b-df2a8a41d1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:53.964497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547f6a6-e018-45c8-aa4b-df2a8a41d1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:54.036020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5547f6a6-e018-45c8-aa4b-df2a8a41d1b8 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:54.037304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5547f6a6-e018-45c8-aa4b-df2a8a41d1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:54.136248 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:54.148323 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 664/1342] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:38:54.443500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59e8de-a1f7-4fb7-80f1-148a718c9793 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:38:54.445044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-da59e8de-a1f7-4fb7-80f1-148a718c9793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:54.559147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59e8de-a1f7-4fb7-80f1-148a718c9793 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:38:54.601610 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:54.660205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60544e3e-e3de-4e1a-a501-a27235285f32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:38:54.660824 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60544e3e-e3de-4e1a-a501-a27235285f32 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:54.661101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60544e3e-e3de-4e1a-a501-a27235285f32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:54.661968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60544e3e-e3de-4e1a-a501-a27235285f32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:54.663631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-60544e3e-e3de-4e1a-a501-a27235285f32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:54.801460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60544e3e-e3de-4e1a-a501-a27235285f32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:38:54.803010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60544e3e-e3de-4e1a-a501-a27235285f32 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:38:54.826977 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 679/1343] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:38:58.211503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae52797-13fa-43c4-8cfc-1a32bd36e413 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:58.212039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae52797-13fa-43c4-8cfc-1a32bd36e413 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:58.212039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae52797-13fa-43c4-8cfc-1a32bd36e413 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:58.213327 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 665/1344] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:38:58.218733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02610d-2553-4ad2-aec0-84db0a387ccf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:58.219211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02610d-2553-4ad2-aec0-84db0a387ccf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:58.219413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02610d-2553-4ad2-aec0-84db0a387ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:58.292741 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-da02610d-2553-4ad2-aec0-84db0a387ccf None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:38:58.293969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-da02610d-2553-4ad2-aec0-84db0a387ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:58.313491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:58.402092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 680/1345] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:38:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:38:58.684712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae52797-13fa-43c4-8cfc-1a32bd36e413 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:38:58.686630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8ae52797-13fa-43c4-8cfc-1a32bd36e413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:58.800476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae52797-13fa-43c4-8cfc-1a32bd36e413 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:38:58.903205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ea466-9756-42df-9df4-190c9521067e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:38:58.903938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ea466-9756-42df-9df4-190c9521067e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:38:58.904079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ea466-9756-42df-9df4-190c9521067e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:38:58.904348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ea466-9756-42df-9df4-190c9521067e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:38:58.905976 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b05ea466-9756-42df-9df4-190c9521067e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:38:58.957945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:38:59.039021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ea466-9756-42df-9df4-190c9521067e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:38:59.040635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ea466-9756-42df-9df4-190c9521067e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:38:59.064932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 666/1346] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:38:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:39:03.790062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c13a0-6476-4fec-affb-a257edd0659c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:03.790635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c13a0-6476-4fec-affb-a257edd0659c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:03.790635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c13a0-6476-4fec-affb-a257edd0659c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:03.791978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 681/1347] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:03.797775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772ac22-935f-43fd-94da-3ab3f46c8931 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:03.798474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772ac22-935f-43fd-94da-3ab3f46c8931 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:03.798474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772ac22-935f-43fd-94da-3ab3f46c8931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:03.884160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7772ac22-935f-43fd-94da-3ab3f46c8931 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:03.885189 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7772ac22-935f-43fd-94da-3ab3f46c8931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:03.989762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:03.992511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 667/1348] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:39:04.280508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c13a0-6476-4fec-affb-a257edd0659c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:04.282240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1a8c13a0-6476-4fec-affb-a257edd0659c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:04.431037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c13a0-6476-4fec-affb-a257edd0659c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:04.486460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:04.514837 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed09766-0722-4c5d-a78b-1dd1defd8e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:04.515517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed09766-0722-4c5d-a78b-1dd1defd8e54 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:04.515790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed09766-0722-4c5d-a78b-1dd1defd8e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:04.516057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed09766-0722-4c5d-a78b-1dd1defd8e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:04.517712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7ed09766-0722-4c5d-a78b-1dd1defd8e54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:04.659613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed09766-0722-4c5d-a78b-1dd1defd8e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:04.661087 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed09766-0722-4c5d-a78b-1dd1defd8e54 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:04.682838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 682/1349] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:39:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:39:10.519202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d7a31-2c63-4c88-8591-aeb03045c479 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:10.519860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d7a31-2c63-4c88-8591-aeb03045c479 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:10.521666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d7a31-2c63-4c88-8591-aeb03045c479 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:10.525344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 668/1350] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:39:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:10.531419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0077b-f442-44a5-96a0-eee2b99e568d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:10.532194 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0077b-f442-44a5-96a0-eee2b99e568d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:10.532194 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0077b-f442-44a5-96a0-eee2b99e568d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:10.669761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b7b0077b-f442-44a5-96a0-eee2b99e568d None None] MFA Rules not processed for user `8d03db170fb6460794e32669456bcdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:10.671098 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b7b0077b-f442-44a5-96a0-eee2b99e568d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:10.724513 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:10.927768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 683/1351] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:39:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:39:15.821434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a052c-186f-451d-bd38-0945f9a64eec None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:15.822558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a052c-186f-451d-bd38-0945f9a64eec None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:15.822558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a052c-186f-451d-bd38-0945f9a64eec None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:15.823735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 669/1352] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:39:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:15.831215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d312e3-d116-413c-9e44-a8515c20da96 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:15.831785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d312e3-d116-413c-9e44-a8515c20da96 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:15.832302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d312e3-d116-413c-9e44-a8515c20da96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:15.956048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-94d312e3-d116-413c-9e44-a8515c20da96 None None] MFA Rules not processed for user `8d03db170fb6460794e32669456bcdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:15.960209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-94d312e3-d116-413c-9e44-a8515c20da96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:16.058843 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:16.103871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 684/1353] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:39:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:39:17.329775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3049f27-2dfd-4d1f-be74-d7a352a7b2c7 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:17.330367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3049f27-2dfd-4d1f-be74-d7a352a7b2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:17.330486 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3049f27-2dfd-4d1f-be74-d7a352a7b2c7 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:17.332020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 670/1354] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:17.337689 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92bb68-d7a1-4a21-8d81-c92871f0cf9c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:17.338473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92bb68-d7a1-4a21-8d81-c92871f0cf9c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:17.338473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92bb68-d7a1-4a21-8d81-c92871f0cf9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:17.434486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3b92bb68-d7a1-4a21-8d81-c92871f0cf9c None None] MFA Rules not processed for user `8d03db170fb6460794e32669456bcdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:17.435713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3b92bb68-d7a1-4a21-8d81-c92871f0cf9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:17.557278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 685/1355] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:39:21.588407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca082b-c907-4ad9-b638-578241e18d28 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:21.589250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca082b-c907-4ad9-b638-578241e18d28 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:21.589465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca082b-c907-4ad9-b638-578241e18d28 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:21.592653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 671/1356] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:21.597598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697c700-b11a-46ff-aa6f-3a7d1ef782d5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:21.598756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697c700-b11a-46ff-aa6f-3a7d1ef782d5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:21.598756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697c700-b11a-46ff-aa6f-3a7d1ef782d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:21.659152 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:21.725471 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8697c700-b11a-46ff-aa6f-3a7d1ef782d5 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:21.726624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8697c700-b11a-46ff-aa6f-3a7d1ef782d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:21.876923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 686/1357] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:39:22.183787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ad059-4867-4081-bf1b-8ceb705c614e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:22.184338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ad059-4867-4081-bf1b-8ceb705c614e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:22.184758 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ad059-4867-4081-bf1b-8ceb705c614e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:22.186624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 672/1358] 10.222.0.44 () {54 vars in 962 bytes} [Sun Sep 11 08:39:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:22.200139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0c6f-5016-40d8-b1c9-17bcd266c34e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:22.200775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0c6f-5016-40d8-b1c9-17bcd266c34e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:22.201372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0c6f-5016-40d8-b1c9-17bcd266c34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:22.306645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-01dc0c6f-5016-40d8-b1c9-17bcd266c34e None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:22.308172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-01dc0c6f-5016-40d8-b1c9-17bcd266c34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:22.416286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:22.542976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 687/1359] 10.222.0.44 () {58 vars in 1059 bytes} [Sun Sep 11 08:39:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:39:22.552679 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ad059-4867-4081-bf1b-8ceb705c614e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:22.554233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f06ad059-4867-4081-bf1b-8ceb705c614e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:22.730881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ad059-4867-4081-bf1b-8ceb705c614e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:22.824349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:22.904483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1c425-96a3-419d-99d0-8fa2a5c691fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:22.905229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1c425-96a3-419d-99d0-8fa2a5c691fc service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:22.905468 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1c425-96a3-419d-99d0-8fa2a5c691fc service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:22.905846 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1c425-96a3-419d-99d0-8fa2a5c691fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:22.907914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-53b1c425-96a3-419d-99d0-8fa2a5c691fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:23.041655 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1c425-96a3-419d-99d0-8fa2a5c691fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:23.043434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1c425-96a3-419d-99d0-8fa2a5c691fc service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:23.070752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 673/1360] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:39:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:39:25.829712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7d46b-3fb9-4af0-8edc-d3967374c510 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:25.830329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7d46b-3fb9-4af0-8edc-d3967374c510 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:25.830329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7d46b-3fb9-4af0-8edc-d3967374c510 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:25.832008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 688/1361] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:25.837983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceb2f2-2729-4e02-b8a9-54b3c03126b6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:25.838861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceb2f2-2729-4e02-b8a9-54b3c03126b6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:25.838861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ceb2f2-2729-4e02-b8a9-54b3c03126b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:25.946507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-53ceb2f2-2729-4e02-b8a9-54b3c03126b6 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:25.947517 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-53ceb2f2-2729-4e02-b8a9-54b3c03126b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:26.013273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:26.122220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 674/1362] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:39:26.415006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7d46b-3fb9-4af0-8edc-d3967374c510 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:26.416575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-15f7d46b-3fb9-4af0-8edc-d3967374c510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:26.668839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7d46b-3fb9-4af0-8edc-d3967374c510 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:26.685098 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:26.829835 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb58ef6-8253-47f1-943f-9a245c36c600 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:26.830512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb58ef6-8253-47f1-943f-9a245c36c600 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:26.830937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb58ef6-8253-47f1-943f-9a245c36c600 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:26.831210 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb58ef6-8253-47f1-943f-9a245c36c600 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:26.833510 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4cb58ef6-8253-47f1-943f-9a245c36c600 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:26.970235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb58ef6-8253-47f1-943f-9a245c36c600 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:26.973602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb58ef6-8253-47f1-943f-9a245c36c600 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:27.002388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 689/1363] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:39:29.838009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efe6c2-a3ff-4c04-9dae-3377d7b8b68a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:29.838616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efe6c2-a3ff-4c04-9dae-3377d7b8b68a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:29.838616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efe6c2-a3ff-4c04-9dae-3377d7b8b68a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:29.840267 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 675/1364] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:29.846307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754535a-cf06-403c-985d-0e812c34bc9c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:29.847039 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754535a-cf06-403c-985d-0e812c34bc9c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:29.847039 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754535a-cf06-403c-985d-0e812c34bc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:29.866578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:29.944461 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1754535a-cf06-403c-985d-0e812c34bc9c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:29.947177 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1754535a-cf06-403c-985d-0e812c34bc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:30.071906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 690/1365] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:39:30.387502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f1517-6ce1-4bad-9346-590774c0f2e4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:30.388008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f1517-6ce1-4bad-9346-590774c0f2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:30.388348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f1517-6ce1-4bad-9346-590774c0f2e4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:30.390810 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 676/1366] 10.222.0.44 () {54 vars in 962 bytes} [Sun Sep 11 08:39:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:30.401143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc45797-37cb-438f-98a2-28a624691dcd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:30.402468 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc45797-37cb-438f-98a2-28a624691dcd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:30.402468 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc45797-37cb-438f-98a2-28a624691dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:30.525198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4bc45797-37cb-438f-98a2-28a624691dcd None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:30.526878 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4bc45797-37cb-438f-98a2-28a624691dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:30.586490 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:30.675076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b762d-2bba-4c05-ac46-8c44324bac7d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:30.675262 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b762d-2bba-4c05-ac46-8c44324bac7d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:30.675558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b762d-2bba-4c05-ac46-8c44324bac7d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:30.677448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 677/1367] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:39:30.682784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d412921-8095-484f-924e-950a4ee318b0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:30.683077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d412921-8095-484f-924e-950a4ee318b0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:30.683342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d412921-8095-484f-924e-950a4ee318b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:30.710409 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 691/1368] 10.222.0.44 () {58 vars in 1059 bytes} [Sun Sep 11 08:39:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:39:30.718539 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc45797-37cb-438f-98a2-28a624691dcd None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:30.730786 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc45797-37cb-438f-98a2-28a624691dcd None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:30.770653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4d412921-8095-484f-924e-950a4ee318b0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:30.772172 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4d412921-8095-484f-924e-950a4ee318b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:30.904487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d1a62-9adc-40ae-ae5b-939b912b3df1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:30.905327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d1a62-9adc-40ae-ae5b-939b912b3df1 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:30.905840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d1a62-9adc-40ae-ae5b-939b912b3df1 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:30.906389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 678/1369] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:39:30.910527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d1a62-9adc-40ae-ae5b-939b912b3df1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:30.913289 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-480d1a62-9adc-40ae-ae5b-939b912b3df1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:30.934333 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:31.057232 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d1a62-9adc-40ae-ae5b-939b912b3df1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:31.061367 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d1a62-9adc-40ae-ae5b-939b912b3df1 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:31.100806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 692/1370] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:39:31.211750 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dfa24-584e-4b06-8a06-6ea92ad86c10 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:31.212331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dfa24-584e-4b06-8a06-6ea92ad86c10 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:31.212682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dfa24-584e-4b06-8a06-6ea92ad86c10 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:31.214278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 679/1371] 10.222.0.44 () {54 vars in 964 bytes} [Sun Sep 11 08:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:31.219816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc611b-f6b3-4d26-b9c9-8d47ad884ca7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:31.220192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc611b-f6b3-4d26-b9c9-8d47ad884ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:31.220526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc611b-f6b3-4d26-b9c9-8d47ad884ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:31.336069 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9adc611b-f6b3-4d26-b9c9-8d47ad884ca7 None None] MFA Rules not processed for user `434c9e87bda343659d2f410a6cdbf805`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:31.337360 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9adc611b-f6b3-4d26-b9c9-8d47ad884ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:31.439294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 693/1372] 10.222.0.44 () {58 vars in 1061 bytes} [Sun Sep 11 08:39:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:39:31.454056 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dfa24-584e-4b06-8a06-6ea92ad86c10 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:31.457548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ec3dfa24-584e-4b06-8a06-6ea92ad86c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:31.503835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:31.661844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dfa24-584e-4b06-8a06-6ea92ad86c10 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:31.787440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e53606-e14c-42fe-b709-a1a75eaa43eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:31.788124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e53606-e14c-42fe-b709-a1a75eaa43eb service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:31.788455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e53606-e14c-42fe-b709-a1a75eaa43eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:31.788836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e53606-e14c-42fe-b709-a1a75eaa43eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:31.790785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-83e53606-e14c-42fe-b709-a1a75eaa43eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:31.885877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:31.960298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e53606-e14c-42fe-b709-a1a75eaa43eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:31.963391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e53606-e14c-42fe-b709-a1a75eaa43eb service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:31.991064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 680/1373] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:39:34.158263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5b764-8956-41c2-b381-24f6207eb7ae None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:34.159293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5b764-8956-41c2-b381-24f6207eb7ae None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:34.159574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5b764-8956-41c2-b381-24f6207eb7ae None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:34.161009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 694/1374] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:34.168428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b683a-d272-4054-bb39-64de64fea569 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:34.169902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b683a-d272-4054-bb39-64de64fea569 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:34.170398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b683a-d272-4054-bb39-64de64fea569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:34.284547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-456b683a-d272-4054-bb39-64de64fea569 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:34.285861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-456b683a-d272-4054-bb39-64de64fea569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:34.416078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 681/1375] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:39:34.734841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5b764-8956-41c2-b381-24f6207eb7ae None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:34.737118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-41f5b764-8956-41c2-b381-24f6207eb7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:34.749647 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:34.961633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5b764-8956-41c2-b381-24f6207eb7ae None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:35.073254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:35.140289 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2c8f3-16f8-4d2b-a2cb-67b9a4d8deb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:35.141155 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c8f3-16f8-4d2b-a2cb-67b9a4d8deb5 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:35.141552 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c8f3-16f8-4d2b-a2cb-67b9a4d8deb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:35.142181 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c8f3-16f8-4d2b-a2cb-67b9a4d8deb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:35.144934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-92e2c8f3-16f8-4d2b-a2cb-67b9a4d8deb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:35.254691 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2c8f3-16f8-4d2b-a2cb-67b9a4d8deb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:35.256286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e2c8f3-16f8-4d2b-a2cb-67b9a4d8deb5 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:35.275362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 695/1376] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:39:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:39:37.682308 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a89ca-4ae3-4215-bfa2-c0bb6fbf1db3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:37.682945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a89ca-4ae3-4215-bfa2-c0bb6fbf1db3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:37.683152 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a89ca-4ae3-4215-bfa2-c0bb6fbf1db3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:37.684944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 682/1377] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:37.690582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4f5d4-1b0b-45c7-bbda-be329e46eac0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:37.691211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4f5d4-1b0b-45c7-bbda-be329e46eac0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:37.691403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4f5d4-1b0b-45c7-bbda-be329e46eac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:37.774961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-09e4f5d4-1b0b-45c7-bbda-be329e46eac0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:37.776347 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-09e4f5d4-1b0b-45c7-bbda-be329e46eac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:37.844901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:37.942965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 696/1378] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:39:38.365760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a89ca-4ae3-4215-bfa2-c0bb6fbf1db3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:38.367984 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b97a89ca-4ae3-4215-bfa2-c0bb6fbf1db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:38.419948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:38.586828 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a89ca-4ae3-4215-bfa2-c0bb6fbf1db3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:38.766249 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:38.786760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1f53a-3519-46e6-8860-e89ded13e249 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:38.787922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1f53a-3519-46e6-8860-e89ded13e249 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:38.788397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1f53a-3519-46e6-8860-e89ded13e249 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:38.788942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1f53a-3519-46e6-8860-e89ded13e249 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:38.790973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-06e1f53a-3519-46e6-8860-e89ded13e249 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:38.929311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1f53a-3519-46e6-8860-e89ded13e249 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:38.931514 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1f53a-3519-46e6-8860-e89ded13e249 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:38.958527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 683/1379] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:39:41.539817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4b957-9cf4-4c8a-8fcd-07d95024267f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:41.540383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4b957-9cf4-4c8a-8fcd-07d95024267f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:41.540383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4b957-9cf4-4c8a-8fcd-07d95024267f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:41.541719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 697/1380] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:41.549331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889bb60-198a-41f7-b2a5-0eb7edca906b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:41.549331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889bb60-198a-41f7-b2a5-0eb7edca906b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:41.551037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889bb60-198a-41f7-b2a5-0eb7edca906b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:41.635588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d889bb60-198a-41f7-b2a5-0eb7edca906b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:41.637067 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d889bb60-198a-41f7-b2a5-0eb7edca906b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:41.718809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:41.767807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 684/1381] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:39:42.065056 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4b957-9cf4-4c8a-8fcd-07d95024267f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:42.066655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-83e4b957-9cf4-4c8a-8fcd-07d95024267f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:42.240561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4b957-9cf4-4c8a-8fcd-07d95024267f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:42.280795 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:42.403199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16328727-424e-4386-9788-5100393683d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:42.403732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16328727-424e-4386-9788-5100393683d1 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:42.404006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16328727-424e-4386-9788-5100393683d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:42.404269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16328727-424e-4386-9788-5100393683d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:42.405892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-16328727-424e-4386-9788-5100393683d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:42.505735 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16328727-424e-4386-9788-5100393683d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:42.507394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16328727-424e-4386-9788-5100393683d1 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:42.527725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 698/1382] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:39:44.881317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f143c-888a-4bf4-9919-e0b80e59f8ec None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:44.881855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f143c-888a-4bf4-9919-e0b80e59f8ec None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:44.882038 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f143c-888a-4bf4-9919-e0b80e59f8ec None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:44.883911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 685/1383] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:44.889766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee21f44-1521-4c8c-bf32-a058ea1b7d74 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:44.890521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee21f44-1521-4c8c-bf32-a058ea1b7d74 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:44.890521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee21f44-1521-4c8c-bf32-a058ea1b7d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:44.937918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:44.972139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5ee21f44-1521-4c8c-bf32-a058ea1b7d74 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:44.975901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5ee21f44-1521-4c8c-bf32-a058ea1b7d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:45.122664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 699/1384] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:39:45.414246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f143c-888a-4bf4-9919-e0b80e59f8ec None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:45.415806 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cd5f143c-888a-4bf4-9919-e0b80e59f8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:45.618669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f143c-888a-4bf4-9919-e0b80e59f8ec None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:45.689441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:45.769526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3407bb-82a7-4358-8081-a478d8be60db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:45.770099 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3407bb-82a7-4358-8081-a478d8be60db service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:45.770392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3407bb-82a7-4358-8081-a478d8be60db service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:45.770659 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3407bb-82a7-4358-8081-a478d8be60db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:45.772317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7d3407bb-82a7-4358-8081-a478d8be60db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:45.896437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3407bb-82a7-4358-8081-a478d8be60db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:45.898599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3407bb-82a7-4358-8081-a478d8be60db service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:45.925983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 686/1385] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:39:48.281614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c769b-7fe4-4209-8b4b-1830a9cc7ab2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:48.282617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c769b-7fe4-4209-8b4b-1830a9cc7ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:48.282617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c769b-7fe4-4209-8b4b-1830a9cc7ab2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:48.283576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 700/1386] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:39:48.290693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c42a1-1afd-4d05-b3d7-83b57f3c56b2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:48.291308 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c42a1-1afd-4d05-b3d7-83b57f3c56b2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:48.291713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c42a1-1afd-4d05-b3d7-83b57f3c56b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:48.398699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ea5c42a1-1afd-4d05-b3d7-83b57f3c56b2 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:48.400825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ea5c42a1-1afd-4d05-b3d7-83b57f3c56b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:48.404268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:48.570557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 687/1387] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:39:48.877500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c769b-7fe4-4209-8b4b-1830a9cc7ab2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:48.879776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-174c769b-7fe4-4209-8b4b-1830a9cc7ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:49.015930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:49.048350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a5674-9033-49cf-8cbf-ac96b9d11b85 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:49.048640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a5674-9033-49cf-8cbf-ac96b9d11b85 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:49.048886 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a5674-9033-49cf-8cbf-ac96b9d11b85 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:49.050388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 688/1388] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:39:49.056494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695fc83-efad-44c4-839c-b0dfd48562dc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:49.056768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695fc83-efad-44c4-839c-b0dfd48562dc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:49.057044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695fc83-efad-44c4-839c-b0dfd48562dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:49.064649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c769b-7fe4-4209-8b4b-1830a9cc7ab2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:49.146960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2695fc83-efad-44c4-839c-b0dfd48562dc None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:49.148103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2695fc83-efad-44c4-839c-b0dfd48562dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:49.224756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434a16b-3739-4915-a7e1-534e33f8573a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:49.225773 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434a16b-3739-4915-a7e1-534e33f8573a service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:49.226046 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434a16b-3739-4915-a7e1-534e33f8573a service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:49.226551 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434a16b-3739-4915-a7e1-534e33f8573a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:49.228440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d434a16b-3739-4915-a7e1-534e33f8573a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:49.272881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:49.297874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 689/1389] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:39:49.333036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:49.360668 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434a16b-3739-4915-a7e1-534e33f8573a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:49.362329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434a16b-3739-4915-a7e1-534e33f8573a service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:49.384069 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 701/1390] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:39:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:39:49.623315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eeff1-6c5c-4f9e-a5f3-350b178b85e8 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:49.623782 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eeff1-6c5c-4f9e-a5f3-350b178b85e8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:49.624253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eeff1-6c5c-4f9e-a5f3-350b178b85e8 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:49.626011 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 690/1391] 10.222.0.44 () {54 vars in 966 bytes} [Sun Sep 11 08:39:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:49.633148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d8a14-2676-4d0d-8765-eeebaa055953 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:49.633518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d8a14-2676-4d0d-8765-eeebaa055953 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:49.633695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d8a14-2676-4d0d-8765-eeebaa055953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:49.748052 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cf5d8a14-2676-4d0d-8765-eeebaa055953 None None] MFA Rules not processed for user `7b86978df61245ae88fead154193fcd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:49.749438 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cf5d8a14-2676-4d0d-8765-eeebaa055953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:49.862055 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 702/1392] 10.222.0.44 () {58 vars in 1063 bytes} [Sun Sep 11 08:39:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:39:49.887293 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927eeff1-6c5c-4f9e-a5f3-350b178b85e8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:49.897844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-927eeff1-6c5c-4f9e-a5f3-350b178b85e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:50.063592 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927eeff1-6c5c-4f9e-a5f3-350b178b85e8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:50.153371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14110b-805e-41fa-8d4e-cb25a84fdeb8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b86978df61245ae88fead154193fcd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:50.153986 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14110b-805e-41fa-8d4e-cb25a84fdeb8 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:50.154288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14110b-805e-41fa-8d4e-cb25a84fdeb8 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:50.154555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14110b-805e-41fa-8d4e-cb25a84fdeb8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:50.156144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8d14110b-805e-41fa-8d4e-cb25a84fdeb8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:50.205209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:50.308580 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14110b-805e-41fa-8d4e-cb25a84fdeb8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:50.310497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14110b-805e-41fa-8d4e-cb25a84fdeb8 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:50.332947 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 691/1393] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:39:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:39:51.369426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640f3b-bc8f-4b1d-88f4-105c3657f037 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:51.370338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640f3b-bc8f-4b1d-88f4-105c3657f037 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:51.370338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640f3b-bc8f-4b1d-88f4-105c3657f037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:51.494190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-96640f3b-bc8f-4b1d-88f4-105c3657f037 None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:51.496320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-96640f3b-bc8f-4b1d-88f4-105c3657f037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:51.523914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:51.666744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 703/1394] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:39:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:39:51.731134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410177c6-b7f1-4859-ade7-81429e4a3de1 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:51.731944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410177c6-b7f1-4859-ade7-81429e4a3de1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:51.732224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410177c6-b7f1-4859-ade7-81429e4a3de1 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:51.733834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 692/1395] 10.222.0.44 () {54 vars in 943 bytes} [Sun Sep 11 08:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:51.745569 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b76ad-e6b5-4ee0-b40e-4471ea129ce4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:51.745719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b76ad-e6b5-4ee0-b40e-4471ea129ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:51.745882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b76ad-e6b5-4ee0-b40e-4471ea129ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:51.860458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b07b76ad-e6b5-4ee0-b40e-4471ea129ce4 None None] MFA Rules not processed for user `e60a2198022c4f979a6e5e9057558c33`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:51.862388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b07b76ad-e6b5-4ee0-b40e-4471ea129ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:51.998279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:52.023583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 704/1396] 10.222.0.44 () {58 vars in 1040 bytes} [Sun Sep 11 08:39:51 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:39:52.036063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410177c6-b7f1-4859-ade7-81429e4a3de1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:52.039549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-410177c6-b7f1-4859-ade7-81429e4a3de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:52.131001 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d958a19-e137-4ae5-8e9d-709323b6c0e4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:52.131529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d958a19-e137-4ae5-8e9d-709323b6c0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:52.132055 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d958a19-e137-4ae5-8e9d-709323b6c0e4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:52.134804 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 705/1397] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:39:52.141351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c3d25-c6f9-4258-b4df-945e4ef980e0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:52.142002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c3d25-c6f9-4258-b4df-945e4ef980e0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:52.142493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c3d25-c6f9-4258-b4df-945e4ef980e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:52.218800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410177c6-b7f1-4859-ade7-81429e4a3de1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:52.227898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:52.237325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7a1c3d25-c6f9-4258-b4df-945e4ef980e0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:52.239163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7a1c3d25-c6f9-4258-b4df-945e4ef980e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:52.348155 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5a872-4251-40c9-978f-0fe09a6e9fee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:52.348763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5a872-4251-40c9-978f-0fe09a6e9fee service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:52.349045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5a872-4251-40c9-978f-0fe09a6e9fee service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:52.349314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5a872-4251-40c9-978f-0fe09a6e9fee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:52.351358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-efb5a872-4251-40c9-978f-0fe09a6e9fee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:52.370443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 706/1398] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:39:52.460072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5a872-4251-40c9-978f-0fe09a6e9fee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:52.461744 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5a872-4251-40c9-978f-0fe09a6e9fee service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:52.470944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 693/1399] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:39:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:39:52.668100 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c3d25-c6f9-4258-b4df-945e4ef980e0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:52.669315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7a1c3d25-c6f9-4258-b4df-945e4ef980e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:52.742546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:52.799366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c3d25-c6f9-4258-b4df-945e4ef980e0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:52.960338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e419b21-3161-4a08-8ff3-5fa36aeaac92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:52.961177 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e419b21-3161-4a08-8ff3-5fa36aeaac92 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:52.961647 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e419b21-3161-4a08-8ff3-5fa36aeaac92 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:52.961969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e419b21-3161-4a08-8ff3-5fa36aeaac92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:52.964433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9e419b21-3161-4a08-8ff3-5fa36aeaac92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:53.059350 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:53.113348 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e419b21-3161-4a08-8ff3-5fa36aeaac92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:53.115600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e419b21-3161-4a08-8ff3-5fa36aeaac92 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:53.148875 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 707/1400] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:39:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:39:55.859368 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0040b0-2b7c-46a8-b7a8-356ec813cae5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:55.859913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0040b0-2b7c-46a8-b7a8-356ec813cae5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:55.860319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0040b0-2b7c-46a8-b7a8-356ec813cae5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:55.869667 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 694/1401] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:55 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:39:55.873617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea3c7f-1895-494e-9630-de1ce22863f7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:55.873617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea3c7f-1895-494e-9630-de1ce22863f7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:55.873617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea3c7f-1895-494e-9630-de1ce22863f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:55.999607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3bea3c7f-1895-494e-9630-de1ce22863f7 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:56.000841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3bea3c7f-1895-494e-9630-de1ce22863f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:56.133591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 708/1402] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:39:56.472508 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0040b0-2b7c-46a8-b7a8-356ec813cae5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:39:56.474706 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ca0040b0-2b7c-46a8-b7a8-356ec813cae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:56.541125 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:56.677471 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0040b0-2b7c-46a8-b7a8-356ec813cae5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:39:56.827016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:56.829388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1f5fb-b029-47d1-b7b5-4556b6db7e10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:39:56.830098 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1f5fb-b029-47d1-b7b5-4556b6db7e10 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:56.830464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1f5fb-b029-47d1-b7b5-4556b6db7e10 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:56.830822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1f5fb-b029-47d1-b7b5-4556b6db7e10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:56.833127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-03d1f5fb-b029-47d1-b7b5-4556b6db7e10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:56.944140 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1f5fb-b029-47d1-b7b5-4556b6db7e10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:39:56.945796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1f5fb-b029-47d1-b7b5-4556b6db7e10 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:39:56.967156 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 695/1403] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:39:59.447362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f35775-df1a-42b7-9cfd-6398a976d9c4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:59.448306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f35775-df1a-42b7-9cfd-6398a976d9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:59.448723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f35775-df1a-42b7-9cfd-6398a976d9c4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:59.450789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 709/1404] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:39:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:39:59.458833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821b8a-66a7-43bb-9a26-3eaae3e2571e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:39:59.459605 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821b8a-66a7-43bb-9a26-3eaae3e2571e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:39:59.459605 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c821b8a-66a7-43bb-9a26-3eaae3e2571e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:39:59.563551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4c821b8a-66a7-43bb-9a26-3eaae3e2571e None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:39:59.565779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4c821b8a-66a7-43bb-9a26-3eaae3e2571e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:39:59.705145 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:39:59.770205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 696/1405] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:00.120843 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f35775-df1a-42b7-9cfd-6398a976d9c4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:00.122517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d0f35775-df1a-42b7-9cfd-6398a976d9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:00.180014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:00.349107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f35775-df1a-42b7-9cfd-6398a976d9c4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:00.501503 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:00.504077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d435b-2b1e-43d8-8f5e-f418da454945 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:00.504771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d435b-2b1e-43d8-8f5e-f418da454945 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:00.505054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d435b-2b1e-43d8-8f5e-f418da454945 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:00.505328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d435b-2b1e-43d8-8f5e-f418da454945 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:00.507049 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-281d435b-2b1e-43d8-8f5e-f418da454945 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:00.618965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d435b-2b1e-43d8-8f5e-f418da454945 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:00.620828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d435b-2b1e-43d8-8f5e-f418da454945 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:00.642106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 710/1406] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:03.370200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0fe91-b147-4389-9366-f84ef10f8a0f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:03.370200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0fe91-b147-4389-9366-f84ef10f8a0f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:03.370200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0fe91-b147-4389-9366-f84ef10f8a0f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:03.370200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 697/1407] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:03.373140 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6e00f-f251-4319-8643-dce4dc60eefb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:03.373568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6e00f-f251-4319-8643-dce4dc60eefb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:03.373758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6e00f-f251-4319-8643-dce4dc60eefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:03.470064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-16e6e00f-f251-4319-8643-dce4dc60eefb None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:03.470064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-16e6e00f-f251-4319-8643-dce4dc60eefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:03.551103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:03.583899 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 711/1408] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:03.882660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0fe91-b147-4389-9366-f84ef10f8a0f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:03.884182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-65c0fe91-b147-4389-9366-f84ef10f8a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:04.069775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0fe91-b147-4389-9366-f84ef10f8a0f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:04.131069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:04.216488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3967ae5-486d-43b1-8c05-7a924d2fc5bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:04.217103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3967ae5-486d-43b1-8c05-7a924d2fc5bf service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:04.217376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3967ae5-486d-43b1-8c05-7a924d2fc5bf service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:04.217640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3967ae5-486d-43b1-8c05-7a924d2fc5bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:04.219291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a3967ae5-486d-43b1-8c05-7a924d2fc5bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:04.341470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3967ae5-486d-43b1-8c05-7a924d2fc5bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:04.343148 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3967ae5-486d-43b1-8c05-7a924d2fc5bf service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:04.366753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 698/1409] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:13.589067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=10 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:14.363930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=72 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:17.398546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4ea03-0f4b-4ce4-b1df-44677e9dd38d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:17.399118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4ea03-0f4b-4ce4-b1df-44677e9dd38d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:17.399118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4ea03-0f4b-4ce4-b1df-44677e9dd38d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:17.400492 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 712/1410] 10.222.0.44 () {54 vars in 918 bytes} [Sun Sep 11 08:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:17.405738 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8a4b4-7f98-471e-be4d-685f095a5a13 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:17.406209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8a4b4-7f98-471e-be4d-685f095a5a13 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:17.406316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8a4b4-7f98-471e-be4d-685f095a5a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:17.495124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4fd8a4b4-7f98-471e-be4d-685f095a5a13 None None] MFA Rules not processed for user `8d03db170fb6460794e32669456bcdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:17.496361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4fd8a4b4-7f98-471e-be4d-685f095a5a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:17.602060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 699/1411] 10.222.0.44 () {58 vars in 1015 bytes} [Sun Sep 11 08:40:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:17.616976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375dfd9-4b7a-4d5c-b945-f56a8096a467 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:17.617261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375dfd9-4b7a-4d5c-b945-f56a8096a467 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:17.617533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375dfd9-4b7a-4d5c-b945-f56a8096a467 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:17.618914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 713/1412] 10.222.0.44 () {54 vars in 918 bytes} [Sun Sep 11 08:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:17.624081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6772a0-ebf7-4c10-91a1-004756ec0d12 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:17.624368 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6772a0-ebf7-4c10-91a1-004756ec0d12 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:17.624641 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6772a0-ebf7-4c10-91a1-004756ec0d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:17.700327 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:17.703780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fa6772a0-ebf7-4c10-91a1-004756ec0d12 None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:17.705023 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa6772a0-ebf7-4c10-91a1-004756ec0d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:17.829195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 700/1413] 10.222.0.44 () {58 vars in 1015 bytes} [Sun Sep 11 08:40:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:20.441300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae5f99-5246-4ce5-b7fd-72582928becc None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:20.441887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae5f99-5246-4ce5-b7fd-72582928becc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:20.441887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae5f99-5246-4ce5-b7fd-72582928becc None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:20.443194 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:20.448954 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bdae6-2caf-4751-bb82-e69e8a823783 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:20.449473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bdae6-2caf-4751-bb82-e69e8a823783 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:20.449566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bdae6-2caf-4751-bb82-e69e8a823783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:20.517019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0e1bdae6-2caf-4751-bb82-e69e8a823783 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:20.518287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0e1bdae6-2caf-4751-bb82-e69e8a823783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:20.573024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:20.617774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:40:20.911359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae5f99-5246-4ce5-b7fd-72582928becc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:20.912885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2cae5f99-5246-4ce5-b7fd-72582928becc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:21.064612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae5f99-5246-4ce5-b7fd-72582928becc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:21.138682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:21.179783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59543298-ac9e-47d3-b4f8-3012412f2a2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:21.180385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59543298-ac9e-47d3-b4f8-3012412f2a2c service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:21.180658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59543298-ac9e-47d3-b4f8-3012412f2a2c service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:21.180924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59543298-ac9e-47d3-b4f8-3012412f2a2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:21.182546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-59543298-ac9e-47d3-b4f8-3012412f2a2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:21.281441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59543298-ac9e-47d3-b4f8-3012412f2a2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:21.283207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59543298-ac9e-47d3-b4f8-3012412f2a2c service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:21.303335 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 715/1416] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:22.492375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d6f73-8cf6-48ba-a226-7c657a8637ac None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:22.492964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d6f73-8cf6-48ba-a226-7c657a8637ac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:22.492964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d6f73-8cf6-48ba-a226-7c657a8637ac None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:22.494376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 08:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:22.500798 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74110b50-2774-4011-9f7c-a3f8d2322b3c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:22.501273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74110b50-2774-4011-9f7c-a3f8d2322b3c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:22.501379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74110b50-2774-4011-9f7c-a3f8d2322b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:22.576968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-74110b50-2774-4011-9f7c-a3f8d2322b3c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:22.578198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-74110b50-2774-4011-9f7c-a3f8d2322b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:22.606116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:22.675632 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 08:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:22.826631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d6f73-8cf6-48ba-a226-7c657a8637ac None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:22.828345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a98d6f73-8cf6-48ba-a226-7c657a8637ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:22.977277 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d6f73-8cf6-48ba-a226-7c657a8637ac None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:23.017429 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:23.092476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3da3b-e84e-4803-aaa8-e1cad6ee6f25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:23.092996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3da3b-e84e-4803-aaa8-e1cad6ee6f25 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:23.093269 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3da3b-e84e-4803-aaa8-e1cad6ee6f25 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:23.093534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3da3b-e84e-4803-aaa8-e1cad6ee6f25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:23.095174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-06d3da3b-e84e-4803-aaa8-e1cad6ee6f25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:23.188636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3da3b-e84e-4803-aaa8-e1cad6ee6f25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:23.190942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3da3b-e84e-4803-aaa8-e1cad6ee6f25 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:23.210054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 703/1419] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:24.399810 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9813f-97c9-4132-b4a6-3c4d5ad4babf None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:24.400364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9813f-97c9-4132-b4a6-3c4d5ad4babf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:24.400364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9813f-97c9-4132-b4a6-3c4d5ad4babf None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:24.401682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 08:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:24.408053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e37e6-3720-4154-bd8f-631a7db2dbb8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:24.408651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e37e6-3720-4154-bd8f-631a7db2dbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:24.408651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e37e6-3720-4154-bd8f-631a7db2dbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:24.462873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:24.485030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-158e37e6-3720-4154-bd8f-631a7db2dbb8 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:24.486571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-158e37e6-3720-4154-bd8f-631a7db2dbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:24.584839 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 08:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:24.737903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9813f-97c9-4132-b4a6-3c4d5ad4babf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:24.739439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7ac9813f-97c9-4132-b4a6-3c4d5ad4babf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:24.889678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9813f-97c9-4132-b4a6-3c4d5ad4babf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:24.902034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:25.019245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2071b-37fe-4abc-9e17-dc66db0e8944 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:25.019759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2071b-37fe-4abc-9e17-dc66db0e8944 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:25.020036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2071b-37fe-4abc-9e17-dc66db0e8944 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:25.020304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2071b-37fe-4abc-9e17-dc66db0e8944 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:25.021905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1ba2071b-37fe-4abc-9e17-dc66db0e8944 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:25.115074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2071b-37fe-4abc-9e17-dc66db0e8944 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:25.116691 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2071b-37fe-4abc-9e17-dc66db0e8944 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:25.134261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 718/1422] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:26.280840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0507ef-a666-4a70-bac9-4fa98fcc06e9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:26.281392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0507ef-a666-4a70-bac9-4fa98fcc06e9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:26.281392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0507ef-a666-4a70-bac9-4fa98fcc06e9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:26.282684 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 08:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:26.289322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf0aa6-bba3-4210-aa8a-3f365d0ba1aa None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:26.289867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf0aa6-bba3-4210-aa8a-3f365d0ba1aa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:26.289867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf0aa6-bba3-4210-aa8a-3f365d0ba1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:26.310324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:26.367294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-47bf0aa6-bba3-4210-aa8a-3f365d0ba1aa None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:26.368567 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-47bf0aa6-bba3-4210-aa8a-3f365d0ba1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:26.466477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 08:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:26.616869 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0507ef-a666-4a70-bac9-4fa98fcc06e9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:26.618552 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6a0507ef-a666-4a70-bac9-4fa98fcc06e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:26.735888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:26.772399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0507ef-a666-4a70-bac9-4fa98fcc06e9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:26.886068 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206b1f6-5bc5-4811-aa74-eaebe674355b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:26.886702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206b1f6-5bc5-4811-aa74-eaebe674355b service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:26.886985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206b1f6-5bc5-4811-aa74-eaebe674355b service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:26.887246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206b1f6-5bc5-4811-aa74-eaebe674355b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:26.888845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3206b1f6-5bc5-4811-aa74-eaebe674355b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:26.966728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:26.980744 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3206b1f6-5bc5-4811-aa74-eaebe674355b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:26.982530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3206b1f6-5bc5-4811-aa74-eaebe674355b service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:27.002111 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 706/1425] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:28.137396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bc828-115a-4821-9316-a14c47266d05 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:28.137942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bc828-115a-4821-9316-a14c47266d05 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:28.137942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bc828-115a-4821-9316-a14c47266d05 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:28.139344 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 08:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:28.146184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c1752-fecd-443e-babc-f1a75fc80158 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:28.146670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c1752-fecd-443e-babc-f1a75fc80158 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:28.146779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c1752-fecd-443e-babc-f1a75fc80158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:28.229573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a49c1752-fecd-443e-babc-f1a75fc80158 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:28.231008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a49c1752-fecd-443e-babc-f1a75fc80158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:28.327055 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 08:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:28.478970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bc828-115a-4821-9316-a14c47266d05 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:28.480418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0b2bc828-115a-4821-9316-a14c47266d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:28.568437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:28.632151 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bc828-115a-4821-9316-a14c47266d05 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:28.750785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbb170-ab11-4e7a-be79-fb17d202f4fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:28.751331 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbb170-ab11-4e7a-be79-fb17d202f4fa service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:28.751607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbb170-ab11-4e7a-be79-fb17d202f4fa service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:28.752424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbb170-ab11-4e7a-be79-fb17d202f4fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:28.754065 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-efcbb170-ab11-4e7a-be79-fb17d202f4fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:28.805673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:28.848974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbb170-ab11-4e7a-be79-fb17d202f4fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:28.850627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbb170-ab11-4e7a-be79-fb17d202f4fa service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:28.869569 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 721/1428] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:29.990126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820bded-c748-4e9d-ac32-480ac7a6d0a7 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:29.990760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820bded-c748-4e9d-ac32-480ac7a6d0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:29.990760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820bded-c748-4e9d-ac32-480ac7a6d0a7 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:29.992082 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 08:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:29.998740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795170d6-8e51-4208-9db3-bdf6cc3db9eb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:29.999246 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795170d6-8e51-4208-9db3-bdf6cc3db9eb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:29.999341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795170d6-8e51-4208-9db3-bdf6cc3db9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:30.076990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-795170d6-8e51-4208-9db3-bdf6cc3db9eb None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:30.078239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-795170d6-8e51-4208-9db3-bdf6cc3db9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:30.174216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 08:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:40:30.324842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820bded-c748-4e9d-ac32-480ac7a6d0a7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:30.326412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9820bded-c748-4e9d-ac32-480ac7a6d0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:30.374895 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:30.475138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820bded-c748-4e9d-ac32-480ac7a6d0a7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:30.589067 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a64243-f093-4734-91a2-f6ee7dc649e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:30.589670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a64243-f093-4734-91a2-f6ee7dc649e2 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:30.589979 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a64243-f093-4734-91a2-f6ee7dc649e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:30.590247 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a64243-f093-4734-91a2-f6ee7dc649e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:30.591848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c8a64243-f093-4734-91a2-f6ee7dc649e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:30.599199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:30.684286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a64243-f093-4734-91a2-f6ee7dc649e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:30.685877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a64243-f093-4734-91a2-f6ee7dc649e2 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:30.705806 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 709/1431] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:31.831814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e61c-350c-4cbc-b35d-18c26e5f9e92 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:31.832380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e61c-350c-4cbc-b35d-18c26e5f9e92 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:31.832380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e61c-350c-4cbc-b35d-18c26e5f9e92 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:31.833867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 08:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:31.840541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a036c-98fa-4e0b-98eb-ca7d64ce2caa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:31.841024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a036c-98fa-4e0b-98eb-ca7d64ce2caa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:31.841118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a036c-98fa-4e0b-98eb-ca7d64ce2caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:31.922103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-274a036c-98fa-4e0b-98eb-ca7d64ce2caa None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:31.923361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-274a036c-98fa-4e0b-98eb-ca7d64ce2caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:31.993192 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:32.029150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 08:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:32.179401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4e61c-350c-4cbc-b35d-18c26e5f9e92 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:32.180940 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dce4e61c-350c-4cbc-b35d-18c26e5f9e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:32.200623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:32.333602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4e61c-350c-4cbc-b35d-18c26e5f9e92 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:32.421224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:32.447411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee03e32-bb16-4dc6-acbb-3acbef5d191c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:32.448008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee03e32-bb16-4dc6-acbb-3acbef5d191c service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:32.448281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee03e32-bb16-4dc6-acbb-3acbef5d191c service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:32.448547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee03e32-bb16-4dc6-acbb-3acbef5d191c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:32.450221 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5ee03e32-bb16-4dc6-acbb-3acbef5d191c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:32.541915 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee03e32-bb16-4dc6-acbb-3acbef5d191c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:32.543550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee03e32-bb16-4dc6-acbb-3acbef5d191c service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:32.562521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 724/1434] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:33.684922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a93e8c-a87a-4378-a8e7-3c6a9251ab1b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:33.685490 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a93e8c-a87a-4378-a8e7-3c6a9251ab1b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:33.685490 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a93e8c-a87a-4378-a8e7-3c6a9251ab1b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:33.686873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 08:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:33.695455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc66b3-2daa-4c0d-8c4d-330050012180 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:33.696169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc66b3-2daa-4c0d-8c4d-330050012180 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:33.696169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc66b3-2daa-4c0d-8c4d-330050012180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:33.772800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c2fc66b3-2daa-4c0d-8c4d-330050012180 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:33.773946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c2fc66b3-2daa-4c0d-8c4d-330050012180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:33.815395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:33.876063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 08:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:34.029355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a93e8c-a87a-4378-a8e7-3c6a9251ab1b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:34.030831 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-14a93e8c-a87a-4378-a8e7-3c6a9251ab1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:34.181837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a93e8c-a87a-4378-a8e7-3c6a9251ab1b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:34.239245 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:34.301262 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f42e33-3440-437c-bc79-da3e2e26c9be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:34.301875 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f42e33-3440-437c-bc79-da3e2e26c9be service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:34.302166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f42e33-3440-437c-bc79-da3e2e26c9be service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:34.302452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f42e33-3440-437c-bc79-da3e2e26c9be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:34.304078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-28f42e33-3440-437c-bc79-da3e2e26c9be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:34.396237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f42e33-3440-437c-bc79-da3e2e26c9be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:34.397839 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f42e33-3440-437c-bc79-da3e2e26c9be service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:34.415759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 712/1437] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:35.550812 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f3c3f-2200-4388-bec0-2326a1848a9b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:35.551351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f3c3f-2200-4388-bec0-2326a1848a9b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:35.551351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f3c3f-2200-4388-bec0-2326a1848a9b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:35.552698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 08:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:35.559010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310e679-d9c3-4f04-bbcd-a3f0af471c32 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:35.559454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310e679-d9c3-4f04-bbcd-a3f0af471c32 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:35.559565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310e679-d9c3-4f04-bbcd-a3f0af471c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:35.635332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0310e679-d9c3-4f04-bbcd-a3f0af471c32 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:35.636553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0310e679-d9c3-4f04-bbcd-a3f0af471c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:35.644217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:35.744247 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 08:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:35.893236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f3c3f-2200-4388-bec0-2326a1848a9b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:35.894798 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-992f3c3f-2200-4388-bec0-2326a1848a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:36.048989 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f3c3f-2200-4388-bec0-2326a1848a9b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:36.075990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:36.164535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee1969-965c-4042-9e5e-c0a036c4eaef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:36.165139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee1969-965c-4042-9e5e-c0a036c4eaef service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:36.165414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee1969-965c-4042-9e5e-c0a036c4eaef service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:36.165682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee1969-965c-4042-9e5e-c0a036c4eaef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:36.167334 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-28ee1969-965c-4042-9e5e-c0a036c4eaef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:36.259196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee1969-965c-4042-9e5e-c0a036c4eaef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:36.260816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee1969-965c-4042-9e5e-c0a036c4eaef service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:36.279862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 727/1440] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:38.383098 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26660be8-0696-42a4-9352-91b715f8ecc3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:38.383714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26660be8-0696-42a4-9352-91b715f8ecc3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:38.383714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26660be8-0696-42a4-9352-91b715f8ecc3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:38.385021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:38.390595 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04d900-2c8c-4815-bd88-b4868dc439fa None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:38.391021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04d900-2c8c-4815-bd88-b4868dc439fa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:38.391137 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04d900-2c8c-4815-bd88-b4868dc439fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:38.432989 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:38.465269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3c04d900-2c8c-4815-bd88-b4868dc439fa None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:38.466529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3c04d900-2c8c-4815-bd88-b4868dc439fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:38.567056 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:38.850922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26660be8-0696-42a4-9352-91b715f8ecc3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:38.852752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-26660be8-0696-42a4-9352-91b715f8ecc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:38.956390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26660be8-0696-42a4-9352-91b715f8ecc3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:38.990421 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:39.031408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6746d-0606-41c2-bce1-3ece28ea51ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:39.032019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6746d-0606-41c2-bce1-3ece28ea51ca service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:39.032296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6746d-0606-41c2-bce1-3ece28ea51ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:39.032568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6746d-0606-41c2-bce1-3ece28ea51ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:39.034826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-00b6746d-0606-41c2-bce1-3ece28ea51ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:39.144530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6746d-0606-41c2-bce1-3ece28ea51ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:39.145859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6746d-0606-41c2-bce1-3ece28ea51ca service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:39.165550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 715/1443] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:41.508047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6beb4c-9876-4e29-9e97-24bdbfc549e5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:41.508784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6beb4c-9876-4e29-9e97-24bdbfc549e5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:41.508784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6beb4c-9876-4e29-9e97-24bdbfc549e5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:41.509922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:41.515306 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22e32b-a3ce-4148-9222-7d9a52c1faf7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:41.515763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22e32b-a3ce-4148-9222-7d9a52c1faf7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:41.515863 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22e32b-a3ce-4148-9222-7d9a52c1faf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:41.584849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ac22e32b-a3ce-4148-9222-7d9a52c1faf7 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:41.586079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ac22e32b-a3ce-4148-9222-7d9a52c1faf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:41.599373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:41.685302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:41.978681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6beb4c-9876-4e29-9e97-24bdbfc549e5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:41.980197 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cb6beb4c-9876-4e29-9e97-24bdbfc549e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:42.084472 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6beb4c-9876-4e29-9e97-24bdbfc549e5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:42.098321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:42.161011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e968607-e625-4b3b-9d5d-69a1d5b59fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:42.161512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e968607-e625-4b3b-9d5d-69a1d5b59fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:42.161786 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e968607-e625-4b3b-9d5d-69a1d5b59fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:42.162052 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e968607-e625-4b3b-9d5d-69a1d5b59fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:42.163671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3e968607-e625-4b3b-9d5d-69a1d5b59fe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:42.275105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e968607-e625-4b3b-9d5d-69a1d5b59fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:42.276668 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e968607-e625-4b3b-9d5d-69a1d5b59fe0 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:42.294396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 730/1446] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:40:44.848291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ed7f7-0131-480d-bf44-fcc74e0498e3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:44.848897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ed7f7-0131-480d-bf44-fcc74e0498e3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:44.848897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ed7f7-0131-480d-bf44-fcc74e0498e3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:44.850173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:44.855887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e0d94-c4e5-468e-8a29-b3c8afa594fc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:44.856322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e0d94-c4e5-468e-8a29-b3c8afa594fc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:44.856424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e0d94-c4e5-468e-8a29-b3c8afa594fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:44.905283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:44.925760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b17e0d94-c4e5-468e-8a29-b3c8afa594fc None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:44.927036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b17e0d94-c4e5-468e-8a29-b3c8afa594fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:45.024279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:45.312647 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ed7f7-0131-480d-bf44-fcc74e0498e3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:45.314804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e24ed7f7-0131-480d-bf44-fcc74e0498e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:45.417494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ed7f7-0131-480d-bf44-fcc74e0498e3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:45.457689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:45.490711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495eaf5b-6ef2-4179-a17f-e19d79618296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:45.491317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495eaf5b-6ef2-4179-a17f-e19d79618296 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:45.491592 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495eaf5b-6ef2-4179-a17f-e19d79618296 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:45.491857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495eaf5b-6ef2-4179-a17f-e19d79618296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:45.493442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-495eaf5b-6ef2-4179-a17f-e19d79618296 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:45.602986 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495eaf5b-6ef2-4179-a17f-e19d79618296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:45.604359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495eaf5b-6ef2-4179-a17f-e19d79618296 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:45.623437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 718/1449] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:48.102342 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9169d-c727-43f1-828e-7feb425c0fe5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:48.102886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9169d-c727-43f1-828e-7feb425c0fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:48.102886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9169d-c727-43f1-828e-7feb425c0fe5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:48.104112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:48.109484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdef99-2859-46f6-a42f-d84e3d7259d0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:48.109894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdef99-2859-46f6-a42f-d84e3d7259d0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:48.110026 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdef99-2859-46f6-a42f-d84e3d7259d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:48.196999 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-41bdef99-2859-46f6-a42f-d84e3d7259d0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:48.198241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-41bdef99-2859-46f6-a42f-d84e3d7259d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:48.219652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:48.296784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:48.586693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9169d-c727-43f1-828e-7feb425c0fe5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:48.588317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4bf9169d-c727-43f1-828e-7feb425c0fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:48.696974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9169d-c727-43f1-828e-7feb425c0fe5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:48.720649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:48.774717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89121eb4-91bc-43ee-8b8f-94c3d87a95e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:48.775417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89121eb4-91bc-43ee-8b8f-94c3d87a95e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:48.775692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89121eb4-91bc-43ee-8b8f-94c3d87a95e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:48.775961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89121eb4-91bc-43ee-8b8f-94c3d87a95e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:48.777572 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-89121eb4-91bc-43ee-8b8f-94c3d87a95e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:48.898198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89121eb4-91bc-43ee-8b8f-94c3d87a95e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:48.899687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89121eb4-91bc-43ee-8b8f-94c3d87a95e4 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:48.918224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 733/1452] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:51.216531 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbaa9e-9d80-4dd7-9b7d-c94fb85a6d44 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:51.217064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbaa9e-9d80-4dd7-9b7d-c94fb85a6d44 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:51.217064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbaa9e-9d80-4dd7-9b7d-c94fb85a6d44 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:51.218372 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:51.224416 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d18eb-518f-494a-ae6e-226f5fe73a91 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:51.224874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d18eb-518f-494a-ae6e-226f5fe73a91 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:51.224984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d18eb-518f-494a-ae6e-226f5fe73a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:51.297134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:51.300603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c51d18eb-518f-494a-ae6e-226f5fe73a91 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:51.301821 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c51d18eb-518f-494a-ae6e-226f5fe73a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:51.399072 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:51.683865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbaa9e-9d80-4dd7-9b7d-c94fb85a6d44 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:51.685400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0ebbaa9e-9d80-4dd7-9b7d-c94fb85a6d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:51.791581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbaa9e-9d80-4dd7-9b7d-c94fb85a6d44 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:51.841508 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:51.866643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc42bec-0976-4cf8-a1d5-73ab686524b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:51.872472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc42bec-0976-4cf8-a1d5-73ab686524b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:51.872472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc42bec-0976-4cf8-a1d5-73ab686524b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:51.872472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc42bec-0976-4cf8-a1d5-73ab686524b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:51.872472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6cc42bec-0976-4cf8-a1d5-73ab686524b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:51.983027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc42bec-0976-4cf8-a1d5-73ab686524b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:51.984451 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc42bec-0976-4cf8-a1d5-73ab686524b1 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:52.003019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 721/1455] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:54.292948 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe772d-68da-4e66-b407-791d74d132a6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:54.293487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe772d-68da-4e66-b407-791d74d132a6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:54.293487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe772d-68da-4e66-b407-791d74d132a6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:54.294859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:54.301247 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee144940-edec-4325-b299-7196fd4826b0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:54.301688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee144940-edec-4325-b299-7196fd4826b0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:54.301803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee144940-edec-4325-b299-7196fd4826b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:54.369295 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ee144940-edec-4325-b299-7196fd4826b0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:54.370554 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ee144940-edec-4325-b299-7196fd4826b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:54.400291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:54.469195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:54.756972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe772d-68da-4e66-b407-791d74d132a6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:54.758570 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-99fe772d-68da-4e66-b407-791d74d132a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:54.863023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe772d-68da-4e66-b407-791d74d132a6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:54.892906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:54.939336 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782afc6-8e65-4953-b3fb-fd19d9a81936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:54.939940 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782afc6-8e65-4953-b3fb-fd19d9a81936 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:54.940217 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782afc6-8e65-4953-b3fb-fd19d9a81936 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:54.940484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782afc6-8e65-4953-b3fb-fd19d9a81936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:54.942101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9782afc6-8e65-4953-b3fb-fd19d9a81936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:55.053138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782afc6-8e65-4953-b3fb-fd19d9a81936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:55.054609 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782afc6-8e65-4953-b3fb-fd19d9a81936 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:55.073603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 736/1458] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:40:57.349785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae609e-0e1b-41de-9ea8-7564eef26988 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:57.350417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae609e-0e1b-41de-9ea8-7564eef26988 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:57.350417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae609e-0e1b-41de-9ea8-7564eef26988 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:57.351983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:40:57.357608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87b687-6236-487f-99b9-549310718312 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:57.358082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87b687-6236-487f-99b9-549310718312 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:57.358199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87b687-6236-487f-99b9-549310718312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:57.424153 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3d87b687-6236-487f-99b9-549310718312 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:40:57.425382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3d87b687-6236-487f-99b9-549310718312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:57.433926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:57.526456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:40:57.812454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae609e-0e1b-41de-9ea8-7564eef26988 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:40:57.814676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e0ae609e-0e1b-41de-9ea8-7564eef26988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:57.920603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae609e-0e1b-41de-9ea8-7564eef26988 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:40:57.978400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:40:57.997040 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55f15c-a59d-42d5-9e2a-289686b0e429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:40:57.997643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55f15c-a59d-42d5-9e2a-289686b0e429 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:40:57.997961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55f15c-a59d-42d5-9e2a-289686b0e429 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:40:57.998215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55f15c-a59d-42d5-9e2a-289686b0e429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:40:57.999803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c55f15c-a59d-42d5-9e2a-289686b0e429 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:40:58.109054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55f15c-a59d-42d5-9e2a-289686b0e429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:40:58.111060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55f15c-a59d-42d5-9e2a-289686b0e429 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:40:58.129424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 724/1461] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:41:00.490569 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233113a-9f9f-4f98-a74b-c988072e7e95 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:00.491119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233113a-9f9f-4f98-a74b-c988072e7e95 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:00.491119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233113a-9f9f-4f98-a74b-c988072e7e95 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:00.492452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:41:00.497823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9207b2-a825-4578-8346-2971949608e5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:00.498266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9207b2-a825-4578-8346-2971949608e5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:00.498411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9207b2-a825-4578-8346-2971949608e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:00.566608 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8f9207b2-a825-4578-8346-2971949608e5 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:41:00.567843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8f9207b2-a825-4578-8346-2971949608e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:00.733808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:00.792379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:41:01.075216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233113a-9f9f-4f98-a74b-c988072e7e95 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:41:01.076763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6233113a-9f9f-4f98-a74b-c988072e7e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:01.180970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233113a-9f9f-4f98-a74b-c988072e7e95 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:41:01.218904 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:01.257023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d568d-d0b9-4130-8058-aa770a9a2fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:41:01.257624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d568d-d0b9-4130-8058-aa770a9a2fcc service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:01.257896 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d568d-d0b9-4130-8058-aa770a9a2fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:01.258186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d568d-d0b9-4130-8058-aa770a9a2fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:01.259775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a61d568d-d0b9-4130-8058-aa770a9a2fcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:01.369534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d568d-d0b9-4130-8058-aa770a9a2fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:41:01.370906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d568d-d0b9-4130-8058-aa770a9a2fcc service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:41:01.390748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 739/1464] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:41:03.613655 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70e5e8-d45f-47c9-89aa-2b684f56ca13 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:03.614391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70e5e8-d45f-47c9-89aa-2b684f56ca13 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:03.614391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70e5e8-d45f-47c9-89aa-2b684f56ca13 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:03.615567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:41:03.621236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd55bb-6b7b-4aee-9194-74391b015a19 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:03.621664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd55bb-6b7b-4aee-9194-74391b015a19 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:03.621782 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd55bb-6b7b-4aee-9194-74391b015a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:03.804933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-18dd55bb-6b7b-4aee-9194-74391b015a19 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:41:03.806500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-18dd55bb-6b7b-4aee-9194-74391b015a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:03.823954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:03.904614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:41:04.187278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70e5e8-d45f-47c9-89aa-2b684f56ca13 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:41:04.189393 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6a70e5e8-d45f-47c9-89aa-2b684f56ca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:04.296037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70e5e8-d45f-47c9-89aa-2b684f56ca13 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:41:04.368636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:04.375340 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c038476-0535-40ee-bee5-ec6ccfb6825c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:41:04.382496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c038476-0535-40ee-bee5-ec6ccfb6825c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:04.382496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c038476-0535-40ee-bee5-ec6ccfb6825c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:04.382496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c038476-0535-40ee-bee5-ec6ccfb6825c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:04.382496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3c038476-0535-40ee-bee5-ec6ccfb6825c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:04.492200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c038476-0535-40ee-bee5-ec6ccfb6825c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:41:04.493570 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c038476-0535-40ee-bee5-ec6ccfb6825c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:41:04.513207 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 727/1467] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:41:06.696158 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f17265-3e25-40c1-9025-170f926f6fc6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:06.696717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f17265-3e25-40c1-9025-170f926f6fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:06.696821 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f17265-3e25-40c1-9025-170f926f6fc6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:06.698244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:41:06.703624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee244a8-3259-4134-a0a3-9fbed8ac32a4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:06.704550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee244a8-3259-4134-a0a3-9fbed8ac32a4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:06.704746 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee244a8-3259-4134-a0a3-9fbed8ac32a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:06.774099 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8ee244a8-3259-4134-a0a3-9fbed8ac32a4 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:41:06.775349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8ee244a8-3259-4134-a0a3-9fbed8ac32a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:06.824198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:06.878582 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:41:07.167189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f17265-3e25-40c1-9025-170f926f6fc6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:41:07.168919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-95f17265-3e25-40c1-9025-170f926f6fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:07.281874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f17265-3e25-40c1-9025-170f926f6fc6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:41:07.333134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:07.364259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05589db3-5093-4598-965c-667e409c6a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:41:07.364901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05589db3-5093-4598-965c-667e409c6a8b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:07.365171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05589db3-5093-4598-965c-667e409c6a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:07.365436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05589db3-5093-4598-965c-667e409c6a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:07.367164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-05589db3-5093-4598-965c-667e409c6a8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:07.481886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05589db3-5093-4598-965c-667e409c6a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:41:07.483269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05589db3-5093-4598-965c-667e409c6a8b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:41:07.504981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 742/1470] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:41:16.876930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=22 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:17.502597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=66 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:27.833681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8e487-8a6d-4c82-a11d-e8c6ad469416 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:27.834251 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8e487-8a6d-4c82-a11d-e8c6ad469416 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:27.834251 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8e487-8a6d-4c82-a11d-e8c6ad469416 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:27.835603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 729/1471] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:41:27.840868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b3756-e4a1-4300-b0de-76f85a4b9dde None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:27.841287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b3756-e4a1-4300-b0de-76f85a4b9dde None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:27.841404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b3756-e4a1-4300-b0de-76f85a4b9dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:27.909382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ba0b3756-e4a1-4300-b0de-76f85a4b9dde None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:41:27.910643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ba0b3756-e4a1-4300-b0de-76f85a4b9dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:28.006908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 743/1472] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:41:28.306474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c81b5-6f47-4ca2-ad5a-aacec3f11940 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:28.306758 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c81b5-6f47-4ca2-ad5a-aacec3f11940 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:28.307030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c81b5-6f47-4ca2-ad5a-aacec3f11940 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:28.308380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 730/1473] 10.222.0.44 () {54 vars in 966 bytes} [Sun Sep 11 08:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:41:28.313982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421e6fb-8834-45db-953c-db95f253f1ab None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:28.314288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421e6fb-8834-45db-953c-db95f253f1ab None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:28.314561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421e6fb-8834-45db-953c-db95f253f1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:28.390281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3421e6fb-8834-45db-953c-db95f253f1ab None None] MFA Rules not processed for user `7b86978df61245ae88fead154193fcd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:41:28.391527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3421e6fb-8834-45db-953c-db95f253f1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:28.411672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:28.467157 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 744/1474] 10.222.0.44 () {58 vars in 1063 bytes} [Sun Sep 11 08:41:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:41:28.474928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c81b5-6f47-4ca2-ad5a-aacec3f11940 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:41:28.476478 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-045c81b5-6f47-4ca2-ad5a-aacec3f11940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:28.593746 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c81b5-6f47-4ca2-ad5a-aacec3f11940 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:41:28.673463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a87221-542f-479a-a765-beede8340a7f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b86978df61245ae88fead154193fcd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:41:28.674065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a87221-542f-479a-a765-beede8340a7f service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:28.674369 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a87221-542f-479a-a765-beede8340a7f service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:28.674633 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a87221-542f-479a-a765-beede8340a7f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:28.676216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6a87221-542f-479a-a765-beede8340a7f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:28.731042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:28.786886 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a87221-542f-479a-a765-beede8340a7f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:41:28.788445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a87221-542f-479a-a765-beede8340a7f service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:41:28.807309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 731/1475] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:41:31.089337 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfc14f-297e-4bb9-8b7b-f6172bca1ff3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:31.089901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfc14f-297e-4bb9-8b7b-f6172bca1ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:31.089901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfc14f-297e-4bb9-8b7b-f6172bca1ff3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:31.091244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 745/1476] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:41:31.097109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f34d41-d768-42e1-a0a7-69f4a3820fdd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:31.097570 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f34d41-d768-42e1-a0a7-69f4a3820fdd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:31.097677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f34d41-d768-42e1-a0a7-69f4a3820fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:31.164564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-10f34d41-d768-42e1-a0a7-69f4a3820fdd None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:41:31.165663 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-10f34d41-d768-42e1-a0a7-69f4a3820fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:31.262681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 732/1477] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:41:31.533316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfc14f-297e-4bb9-8b7b-f6172bca1ff3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:41:31.535159 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3ecfc14f-297e-4bb9-8b7b-f6172bca1ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:31.670768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfc14f-297e-4bb9-8b7b-f6172bca1ff3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:41:31.724061 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:31.769510 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a1ac9-ce3b-4f29-9ebe-e76da990e44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:41:31.770122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a1ac9-ce3b-4f29-9ebe-e76da990e44d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:31.770421 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a1ac9-ce3b-4f29-9ebe-e76da990e44d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:31.770690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a1ac9-ce3b-4f29-9ebe-e76da990e44d admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:31.771327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a1ac9-ce3b-4f29-9ebe-e76da990e44d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:41:31.773092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-727a1ac9-ce3b-4f29-9ebe-e76da990e44d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:41:31.773092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Sep 11 08:41:31.773092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: Sep 11 08:41:31.774254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a1ac9-ce3b-4f29-9ebe-e76da990e44d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:41:31.794881 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 746/1478] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 08:41:31 2022] GET /identity/v3/projects => generated 2553 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:41:34.020949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ac1c6-e17b-4b49-b487-cc3cee22720e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:34.021511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ac1c6-e17b-4b49-b487-cc3cee22720e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:34.021511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ac1c6-e17b-4b49-b487-cc3cee22720e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:34.022864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 733/1479] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:41:34.028130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b145112-fae4-4eb0-b675-eded08ceed1f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:34.028583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b145112-fae4-4eb0-b675-eded08ceed1f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:34.028695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b145112-fae4-4eb0-b675-eded08ceed1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:34.093499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9b145112-fae4-4eb0-b675-eded08ceed1f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:41:34.094740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9b145112-fae4-4eb0-b675-eded08ceed1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:34.189905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 747/1480] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:41:34.491642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ac1c6-e17b-4b49-b487-cc3cee22720e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:41:34.493138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-233ac1c6-e17b-4b49-b487-cc3cee22720e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:34.503471 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:34.644561 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ac1c6-e17b-4b49-b487-cc3cee22720e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:41:34.733712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:34.768105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc48b69-10d2-462f-b745-0931b8c8bf29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:41:34.768706 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc48b69-10d2-462f-b745-0931b8c8bf29 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:34.768978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc48b69-10d2-462f-b745-0931b8c8bf29 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:34.769241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc48b69-10d2-462f-b745-0931b8c8bf29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:34.770899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1bc48b69-10d2-462f-b745-0931b8c8bf29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:34.862356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc48b69-10d2-462f-b745-0931b8c8bf29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:41:34.863941 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc48b69-10d2-462f-b745-0931b8c8bf29 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:41:34.882221 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 734/1481] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:41:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:41:37.258352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c6a39-a9f3-4988-b9c1-72f7a1cf9175 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:37.258886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c6a39-a9f3-4988-b9c1-72f7a1cf9175 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:37.258886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c6a39-a9f3-4988-b9c1-72f7a1cf9175 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:37.260142 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 748/1482] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:41:37.266186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6986e0-5063-4804-a3c2-6de6c2d7d786 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:37.266717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6986e0-5063-4804-a3c2-6de6c2d7d786 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:37.266717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6986e0-5063-4804-a3c2-6de6c2d7d786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:37.334606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1a6986e0-5063-4804-a3c2-6de6c2d7d786 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:41:37.335825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1a6986e0-5063-4804-a3c2-6de6c2d7d786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:37.382546 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:37.432547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 735/1483] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:41:37.719284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c6a39-a9f3-4988-b9c1-72f7a1cf9175 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:41:37.720828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-242c6a39-a9f3-4988-b9c1-72f7a1cf9175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:37.745302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:37.824110 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c6a39-a9f3-4988-b9c1-72f7a1cf9175 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:41:37.899095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be30c8-4eab-46c8-8314-cb97aeadb04a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:41:37.899711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be30c8-4eab-46c8-8314-cb97aeadb04a service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:37.899986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be30c8-4eab-46c8-8314-cb97aeadb04a service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:37.900258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be30c8-4eab-46c8-8314-cb97aeadb04a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:37.901878 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-06be30c8-4eab-46c8-8314-cb97aeadb04a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:38.015853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be30c8-4eab-46c8-8314-cb97aeadb04a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:41:38.017354 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be30c8-4eab-46c8-8314-cb97aeadb04a service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:41:38.037500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 749/1484] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:41:38.875314 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:40.769939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed01b8-7754-4f7f-bd3f-fc796f663fde None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:40.770568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed01b8-7754-4f7f-bd3f-fc796f663fde None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:40.770568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed01b8-7754-4f7f-bd3f-fc796f663fde None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:40.771855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 736/1485] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:41:40.777144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bae90-9671-44bc-b6b9-150d5baca2ab None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:40.777580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bae90-9671-44bc-b6b9-150d5baca2ab None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:40.777716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bae90-9671-44bc-b6b9-150d5baca2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:40.845190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-841bae90-9671-44bc-b6b9-150d5baca2ab None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:41:40.846438 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-841bae90-9671-44bc-b6b9-150d5baca2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:40.943350 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 750/1486] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:41:41.235584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed01b8-7754-4f7f-bd3f-fc796f663fde None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:41:41.237117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a7ed01b8-7754-4f7f-bd3f-fc796f663fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:41.341520 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed01b8-7754-4f7f-bd3f-fc796f663fde None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:41:41.419795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:41.422742 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed686d16-6173-4575-8310-0d365d4a8921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:41:41.423662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed686d16-6173-4575-8310-0d365d4a8921 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:41.424076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed686d16-6173-4575-8310-0d365d4a8921 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:41.424483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed686d16-6173-4575-8310-0d365d4a8921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:41.427154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ed686d16-6173-4575-8310-0d365d4a8921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:41.547500 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed686d16-6173-4575-8310-0d365d4a8921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:41:41.548927 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed686d16-6173-4575-8310-0d365d4a8921 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:41:41.567709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 737/1487] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:41:43.960588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fb103-ab08-449e-b1e5-8576504de90f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:43.961359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fb103-ab08-449e-b1e5-8576504de90f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:43.961359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fb103-ab08-449e-b1e5-8576504de90f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:43.962565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 751/1488] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:41:43.969184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506fd95-f971-4d75-bcca-aa1a87c63e8b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:43.969818 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506fd95-f971-4d75-bcca-aa1a87c63e8b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:43.969818 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506fd95-f971-4d75-bcca-aa1a87c63e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:44.043148 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1506fd95-f971-4d75-bcca-aa1a87c63e8b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:41:44.044366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1506fd95-f971-4d75-bcca-aa1a87c63e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:44.104687 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:44.153114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 738/1489] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:41:44.448500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fb103-ab08-449e-b1e5-8576504de90f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:41:44.450047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3c3fb103-ab08-449e-b1e5-8576504de90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:44.501990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:44.581572 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fb103-ab08-449e-b1e5-8576504de90f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:41:44.666807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1417c-6a5b-416a-8030-ad9229f7d839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:41:44.667428 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1417c-6a5b-416a-8030-ad9229f7d839 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:44.667708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1417c-6a5b-416a-8030-ad9229f7d839 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:44.667975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1417c-6a5b-416a-8030-ad9229f7d839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:44.669651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2df1417c-6a5b-416a-8030-ad9229f7d839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:44.776471 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:44.806252 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1417c-6a5b-416a-8030-ad9229f7d839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:41:44.807762 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1417c-6a5b-416a-8030-ad9229f7d839 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:41:44.830771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 752/1490] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:41:47.511414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0222d2-53f2-4faf-9f80-e6afd3c2198a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:47.511844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0222d2-53f2-4faf-9f80-e6afd3c2198a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:47.511942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0222d2-53f2-4faf-9f80-e6afd3c2198a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:47.513322 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 739/1491] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 08:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:41:47.519757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c92a1-73e5-424b-be7a-0e64edcd8852 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:47.520255 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c92a1-73e5-424b-be7a-0e64edcd8852 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:47.520384 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c92a1-73e5-424b-be7a-0e64edcd8852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:47.586686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3d8c92a1-73e5-424b-be7a-0e64edcd8852 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:41:47.587905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3d8c92a1-73e5-424b-be7a-0e64edcd8852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:47.685709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 753/1492] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 08:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:41:47.976180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0222d2-53f2-4faf-9f80-e6afd3c2198a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:41:47.977964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0f0222d2-53f2-4faf-9f80-e6afd3c2198a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:48.082778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0222d2-53f2-4faf-9f80-e6afd3c2198a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:41:48.156957 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ce0c6-3584-41b8-a93e-813ed49c8b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:41:48.157456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ce0c6-3584-41b8-a93e-813ed49c8b7c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:41:48.157735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ce0c6-3584-41b8-a93e-813ed49c8b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:41:48.158003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ce0c6-3584-41b8-a93e-813ed49c8b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:41:48.159662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4b3ce0c6-3584-41b8-a93e-813ed49c8b7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:41:48.168277 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:48.274557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ce0c6-3584-41b8-a93e-813ed49c8b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:41:48.275999 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ce0c6-3584-41b8-a93e-813ed49c8b7c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:41:48.294835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 740/1493] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:41:57.684179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=82 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:41:58.292461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=50 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:43:12.888606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629b7f3-6923-43b9-9430-0a0850c20c40 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:12.889084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629b7f3-6923-43b9-9430-0a0850c20c40 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:12.889188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629b7f3-6923-43b9-9430-0a0850c20c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:12.965281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a629b7f3-6923-43b9-9430-0a0850c20c40 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:43:12.967077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a629b7f3-6923-43b9-9430-0a0850c20c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:13.065304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 754/1494] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:43:13.073007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b24c55-dac9-4674-8ecf-50024073025b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:13.073548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b24c55-dac9-4674-8ecf-50024073025b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:13.073548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b24c55-dac9-4674-8ecf-50024073025b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:13.157017 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-44b24c55-dac9-4674-8ecf-50024073025b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:43:13.168755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-44b24c55-dac9-4674-8ecf-50024073025b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:13.261688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 741/1495] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:43:13.266801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629b7f3-6923-43b9-9430-0a0850c20c40 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:43:13.267656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a629b7f3-6923-43b9-9430-0a0850c20c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:13.279874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629b7f3-6923-43b9-9430-0a0850c20c40 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:43:13.345194 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:43:13.380858 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52745838-5a88-4a51-925c-9e39bd9103b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:43:13.381470 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52745838-5a88-4a51-925c-9e39bd9103b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:13.381743 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52745838-5a88-4a51-925c-9e39bd9103b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:13.382009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52745838-5a88-4a51-925c-9e39bd9103b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:13.382675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52745838-5a88-4a51-925c-9e39bd9103b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:43:13.382971 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52745838-5a88-4a51-925c-9e39bd9103b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:43:13.383984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-52745838-5a88-4a51-925c-9e39bd9103b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:43:13.383984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Sep 11 08:43:13.383984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: Sep 11 08:43:13.384871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52745838-5a88-4a51-925c-9e39bd9103b8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:43:13.397398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 755/1496] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:43:13.406820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b24c55-dac9-4674-8ecf-50024073025b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:43:13.416051 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b24c55-dac9-4674-8ecf-50024073025b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:43:13.479435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:43:13.515892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71afa3a-cc2a-4deb-a0d6-960b39008be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:43:13.516498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71afa3a-cc2a-4deb-a0d6-960b39008be4 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:13.516798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71afa3a-cc2a-4deb-a0d6-960b39008be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:13.517060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71afa3a-cc2a-4deb-a0d6-960b39008be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:13.517697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71afa3a-cc2a-4deb-a0d6-960b39008be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:43:13.519477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-c71afa3a-cc2a-4deb-a0d6-960b39008be4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:43:13.519477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Sep 11 08:43:13.519477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: Sep 11 08:43:13.520643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71afa3a-cc2a-4deb-a0d6-960b39008be4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:43:13.658122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 742/1497] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:43:13 2022] POST /identity/v3/projects => generated 377 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:43:13.662401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52745838-5a88-4a51-925c-9e39bd9103b8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:43:13.663988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-52745838-5a88-4a51-925c-9e39bd9103b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:13.700221 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52745838-5a88-4a51-925c-9e39bd9103b8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:43:13.797271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72fc84-6245-4d40-8a10-01f3534bbe63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:43:13.797875 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72fc84-6245-4d40-8a10-01f3534bbe63 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:13.798165 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72fc84-6245-4d40-8a10-01f3534bbe63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:13.798467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72fc84-6245-4d40-8a10-01f3534bbe63 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:13.799089 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72fc84-6245-4d40-8a10-01f3534bbe63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:43:13.800106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-7c72fc84-6245-4d40-8a10-01f3534bbe63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:43:13.800106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Sep 11 08:43:13.800106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: Sep 11 08:43:13.801308 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72fc84-6245-4d40-8a10-01f3534bbe63 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:43:13.903464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 756/1498] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:43:13 2022] POST /identity/v3/users => generated 289 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:43:13.907720 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71afa3a-cc2a-4deb-a0d6-960b39008be4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:43:13.909327 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c71afa3a-cc2a-4deb-a0d6-960b39008be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:13.977266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71afa3a-cc2a-4deb-a0d6-960b39008be4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:43:14.069566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:43:14.080595 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972850ef-6d68-46a7-93b7-41fde459f163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:43:14.088128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972850ef-6d68-46a7-93b7-41fde459f163 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:14.088128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972850ef-6d68-46a7-93b7-41fde459f163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:14.088128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972850ef-6d68-46a7-93b7-41fde459f163 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:14.088128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972850ef-6d68-46a7-93b7-41fde459f163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:43:14.088128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-972850ef-6d68-46a7-93b7-41fde459f163 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:43:14.088128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Sep 11 08:43:14.088128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: Sep 11 08:43:14.088750 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972850ef-6d68-46a7-93b7-41fde459f163 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:43:14.097861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 743/1499] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:43:14.101991 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72fc84-6245-4d40-8a10-01f3534bbe63 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:43:14.103569 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7c72fc84-6245-4d40-8a10-01f3534bbe63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:14.116296 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72fc84-6245-4d40-8a10-01f3534bbe63 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:43:14.142864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:43:14.214407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bdd156-4563-4b8d-9b3c-74990b0674d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:43:14.214921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdd156-4563-4b8d-9b3c-74990b0674d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:14.215193 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdd156-4563-4b8d-9b3c-74990b0674d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:14.215458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdd156-4563-4b8d-9b3c-74990b0674d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:14.216143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdd156-4563-4b8d-9b3c-74990b0674d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:43:14.217151 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-60bdd156-4563-4b8d-9b3c-74990b0674d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:43:14.217151 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Sep 11 08:43:14.217151 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: Sep 11 08:43:14.218034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdd156-4563-4b8d-9b3c-74990b0674d0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:43:14.230640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 757/1500] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:43:14.235305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972850ef-6d68-46a7-93b7-41fde459f163 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:43:14.236811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-972850ef-6d68-46a7-93b7-41fde459f163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:14.270270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972850ef-6d68-46a7-93b7-41fde459f163 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:43:14.366225 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12506221-6620-4e2e-94cd-d5b7793f5c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:43:14.366880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12506221-6620-4e2e-94cd-d5b7793f5c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:14.367148 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12506221-6620-4e2e-94cd-d5b7793f5c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:14.367412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12506221-6620-4e2e-94cd-d5b7793f5c92 admin admin] PATH_INFO: `/v3/projects/afd2b8ed89084c3faee3440783a78c30/users/dd0f7d0b2eab440a88475a609d1342da/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:14.391360 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12506221-6620-4e2e-94cd-d5b7793f5c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd2b8ed89084c3faee3440783a78c30, user_id=dd0f7d0b2eab440a88475a609d1342da, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:43:14.392398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-12506221-6620-4e2e-94cd-d5b7793f5c92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:43:14.392398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Sep 11 08:43:14.392398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: Sep 11 08:43:14.394344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12506221-6620-4e2e-94cd-d5b7793f5c92 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:43:14.480633 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 744/1501] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:43:14 2022] PUT /identity/v3/projects/afd2b8ed89084c3faee3440783a78c30/users/dd0f7d0b2eab440a88475a609d1342da/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:43:14.493635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2c5f7-f589-4e22-bfa4-bdda0e0ef09b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:14.493918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2c5f7-f589-4e22-bfa4-bdda0e0ef09b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:14.494214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2c5f7-f589-4e22-bfa4-bdda0e0ef09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:14.553720 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e0c2c5f7-f589-4e22-bfa4-bdda0e0ef09b None None] MFA Rules not processed for user `dd0f7d0b2eab440a88475a609d1342da`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:43:14.554984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e0c2c5f7-f589-4e22-bfa4-bdda0e0ef09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:14.634850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 758/1502] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:43:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:43:14.651941 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d16d8-1779-4929-ba0b-672fc50f8cfd None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:14.652247 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d16d8-1779-4929-ba0b-672fc50f8cfd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:14.652555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d16d8-1779-4929-ba0b-672fc50f8cfd None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:14.654529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 745/1503] 10.222.0.44 () {54 vars in 964 bytes} [Sun Sep 11 08:43:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:43:14.660358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191331d-75e2-4720-a341-a6e7e63e4c88 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:14.660732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191331d-75e2-4720-a341-a6e7e63e4c88 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:14.661016 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191331d-75e2-4720-a341-a6e7e63e4c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:14.715848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:43:14.773282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3191331d-75e2-4720-a341-a6e7e63e4c88 None None] MFA Rules not processed for user `434c9e87bda343659d2f410a6cdbf805`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:43:14.774430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3191331d-75e2-4720-a341-a6e7e63e4c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:14.855320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 759/1504] 10.222.0.44 () {58 vars in 1061 bytes} [Sun Sep 11 08:43:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:43:14.865390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d16d8-1779-4929-ba0b-672fc50f8cfd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:43:14.868126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-963d16d8-1779-4929-ba0b-672fc50f8cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:14.926694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:43:14.996950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d16d8-1779-4929-ba0b-672fc50f8cfd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:43:15.094187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afeaf5-98ca-45b7-8190-0b4cc30a4c90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:43:15.094914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afeaf5-98ca-45b7-8190-0b4cc30a4c90 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:15.095144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afeaf5-98ca-45b7-8190-0b4cc30a4c90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:15.095408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afeaf5-98ca-45b7-8190-0b4cc30a4c90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:15.097070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4afeaf5-98ca-45b7-8190-0b4cc30a4c90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:15.216768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afeaf5-98ca-45b7-8190-0b4cc30a4c90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:43:15.219332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afeaf5-98ca-45b7-8190-0b4cc30a4c90 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:43:15.224168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:43:15.240275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 746/1505] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:43:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:43:15.290265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380819ec-6c96-4479-ba91-2d05a60d5cd9 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:15.290887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380819ec-6c96-4479-ba91-2d05a60d5cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:15.290887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380819ec-6c96-4479-ba91-2d05a60d5cd9 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:15.292865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 760/1506] 10.222.0.44 () {52 vars in 830 bytes} [Sun Sep 11 08:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:43:15.308958 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afeaf5-98ca-45b7-8190-0b4cc30a4c90 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:43:15.310938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4afeaf5-98ca-45b7-8190-0b4cc30a4c90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:15.417784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afeaf5-98ca-45b7-8190-0b4cc30a4c90 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:43:15.523297 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:43:15.523923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:15.524194 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:15.524456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] PATH_INFO: `/v3/users/dd0f7d0b2eab440a88475a609d1342da` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:15.537516 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd0f7d0b2eab440a88475a609d1342da)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:43:15.538571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:43:15.538571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Sep 11 08:43:15.538571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: Sep 11 08:43:15.539865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:43:15.551671 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:43:15.800052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd0f7d0b2eab440a88475a609d1342da'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:43:15.827219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd0f7d0b2eab440a88475a609d1342da'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:43:15.907623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd0f7d0b2eab440a88475a609d1342da'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:43:15.908058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd0f7d0b2eab440a88475a609d1342da'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:43:15.925574 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd0f7d0b2eab440a88475a609d1342da'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:43:15.930633 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 747/1507] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:43:15 2022] DELETE /identity/v3/users/dd0f7d0b2eab440a88475a609d1342da => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:43:15.936973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164edeb-a7a9-4301-80c5-23ffac539189 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:15.937258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164edeb-a7a9-4301-80c5-23ffac539189 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:15.939408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164edeb-a7a9-4301-80c5-23ffac539189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:16.003207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a164edeb-a7a9-4301-80c5-23ffac539189 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:43:16.003795 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a164edeb-a7a9-4301-80c5-23ffac539189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:16.103095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:43:16.116317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 761/1508] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:43:16.126064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:43:16.127344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:16.233273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842e326-3d84-46e7-9a49-ad81dd5dbaed admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:43:16.295417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:43:16.315052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774b1e1-b385-41e2-96b6-47715229758b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:43:16.315679 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774b1e1-b385-41e2-96b6-47715229758b service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:16.315956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774b1e1-b385-41e2-96b6-47715229758b service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:16.316230 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774b1e1-b385-41e2-96b6-47715229758b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:16.317862 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4774b1e1-b385-41e2-96b6-47715229758b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:16.439527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774b1e1-b385-41e2-96b6-47715229758b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:43:16.441008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774b1e1-b385-41e2-96b6-47715229758b service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:43:16.460135 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 748/1509] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:43:16.921209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164edeb-a7a9-4301-80c5-23ffac539189 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:43:16.922576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a164edeb-a7a9-4301-80c5-23ffac539189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:43:16.935149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164edeb-a7a9-4301-80c5-23ffac539189 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:43:17.043556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700054df-c81a-441f-9e42-54a4c9eb25b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:43:17.044737 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700054df-c81a-441f-9e42-54a4c9eb25b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:43:17.045011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700054df-c81a-441f-9e42-54a4c9eb25b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:43:17.045278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700054df-c81a-441f-9e42-54a4c9eb25b7 admin admin] PATH_INFO: `/v3/projects/afd2b8ed89084c3faee3440783a78c30` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:43:17.055723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700054df-c81a-441f-9e42-54a4c9eb25b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afd2b8ed89084c3faee3440783a78c30)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:43:17.056788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-700054df-c81a-441f-9e42-54a4c9eb25b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:43:17.056788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Sep 11 08:43:17.056788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: Sep 11 08:43:17.057997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700054df-c81a-441f-9e42-54a4c9eb25b7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:43:17.182059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-700054df-c81a-441f-9e42-54a4c9eb25b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afd2b8ed89084c3faee3440783a78c30', 'request_id': 'req-700054df-c81a-441f-9e42-54a4c9eb25b7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:43:17.250888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-700054df-c81a-441f-9e42-54a4c9eb25b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afd2b8ed89084c3faee3440783a78c30', 'request_id': 'req-700054df-c81a-441f-9e42-54a4c9eb25b7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:43:17.257800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-700054df-c81a-441f-9e42-54a4c9eb25b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afd2b8ed89084c3faee3440783a78c30', 'request_id': 'req-700054df-c81a-441f-9e42-54a4c9eb25b7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:43:17.290034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-700054df-c81a-441f-9e42-54a4c9eb25b7 admin admin] The token cache is being invalidate because project afd2b8ed89084c3faee3440783a78c30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:43:17.290342 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-700054df-c81a-441f-9e42-54a4c9eb25b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:43:17.290964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-700054df-c81a-441f-9e42-54a4c9eb25b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:43:17.293330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 762/1510] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:43:16 2022] DELETE /identity/v3/projects/afd2b8ed89084c3faee3440783a78c30 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:43:26.457899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=62 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:43:27.287791 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:44:41.708153 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb229493-a604-47e2-b850-ae4a4d9672e8 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:41.708746 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb229493-a604-47e2-b850-ae4a4d9672e8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:41.708746 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb229493-a604-47e2-b850-ae4a4d9672e8 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:41.710039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 08:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:44:41.714921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8fc3a-c5ca-492f-8621-3a878a381ff3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:41.715365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8fc3a-c5ca-492f-8621-3a878a381ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:41.715460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8fc3a-c5ca-492f-8621-3a878a381ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:41.795974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e8f8fc3a-c5ca-492f-8621-3a878a381ff3 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:44:41.797192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e8f8fc3a-c5ca-492f-8621-3a878a381ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:44:41.907708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 08:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:44:42.061918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb229493-a604-47e2-b850-ae4a4d9672e8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:44:42.064228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eb229493-a604-47e2-b850-ae4a4d9672e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:44:42.183592 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb229493-a604-47e2-b850-ae4a4d9672e8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:44:42.189589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc840dc9-5718-45ac-8c73-ac6145efe411 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:44:42.190212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc840dc9-5718-45ac-8c73-ac6145efe411 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:42.190488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc840dc9-5718-45ac-8c73-ac6145efe411 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:42.190751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc840dc9-5718-45ac-8c73-ac6145efe411 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:42.198763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc840dc9-5718-45ac-8c73-ac6145efe411 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:44:42.200312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc840dc9-5718-45ac-8c73-ac6145efe411 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:44:42.209694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 750/1513] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:44:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:44:43.298034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc0950-ad0b-454c-b4d9-d0601a55cecf None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:43.298676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc0950-ad0b-454c-b4d9-d0601a55cecf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:43.298676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc0950-ad0b-454c-b4d9-d0601a55cecf None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:43.299970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 08:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:44:43.306463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0acaa1-1c5e-4823-a709-362ff5656251 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:43.306921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0acaa1-1c5e-4823-a709-362ff5656251 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:43.307026 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0acaa1-1c5e-4823-a709-362ff5656251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:43.327030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4f0acaa1-1c5e-4823-a709-362ff5656251 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:44:43.328240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4f0acaa1-1c5e-4823-a709-362ff5656251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:44:43.336512 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 08:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:44:43.387845 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac79168-9b60-4d47-8fba-7a76b5d8fb03 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:43.388132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac79168-9b60-4d47-8fba-7a76b5d8fb03 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:43.388401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac79168-9b60-4d47-8fba-7a76b5d8fb03 None None] PATH_INFO: `/v3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:43.389732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 765/1516] 10.222.0.43 () {54 vars in 925 bytes} [Sun Sep 11 08:44:43 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:44:43.396303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563dfca-5509-43d3-8037-e3fa2e539231 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:43.396583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563dfca-5509-43d3-8037-e3fa2e539231 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:43.396848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563dfca-5509-43d3-8037-e3fa2e539231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:43.457564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6563dfca-5509-43d3-8037-e3fa2e539231 None None] MFA Rules not processed for user `8d03db170fb6460794e32669456bcdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:44:43.458815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6563dfca-5509-43d3-8037-e3fa2e539231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:44:43.489994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac79168-9b60-4d47-8fba-7a76b5d8fb03 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:44:43.504864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac79168-9b60-4d47-8fba-7a76b5d8fb03 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:44:43.510927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7da20-77c7-4ce8-9afa-c1e1a9caed81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:44:43.511543 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7da20-77c7-4ce8-9afa-c1e1a9caed81 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:43.511821 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7da20-77c7-4ce8-9afa-c1e1a9caed81 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:43.512093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7da20-77c7-4ce8-9afa-c1e1a9caed81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:43.516929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:44:43.520988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7da20-77c7-4ce8-9afa-c1e1a9caed81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:44:43.522693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7da20-77c7-4ce8-9afa-c1e1a9caed81 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:44:43.533288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 766/1517] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:44:43.537133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 752/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Sun Sep 11 08:44:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:44:52.340449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828805fe-e54c-4114-8730-5a2ae93690f6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:52.345368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828805fe-e54c-4114-8730-5a2ae93690f6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:52.345368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828805fe-e54c-4114-8730-5a2ae93690f6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:52.345368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 767/1519] 10.222.0.44 () {54 vars in 961 bytes} [Sun Sep 11 08:44:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:44:52.348217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fab69f-b518-4a06-b13b-85e0c7dd70b4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:52.348662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fab69f-b518-4a06-b13b-85e0c7dd70b4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:52.348765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fab69f-b518-4a06-b13b-85e0c7dd70b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:52.371069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c1fab69f-b518-4a06-b13b-85e0c7dd70b4 None None] MFA Rules not processed for user `8d03db170fb6460794e32669456bcdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:44:52.374064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c1fab69f-b518-4a06-b13b-85e0c7dd70b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:44:52.381094 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 753/1520] 10.222.0.44 () {58 vars in 1058 bytes} [Sun Sep 11 08:44:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:44:52.388939 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828805fe-e54c-4114-8730-5a2ae93690f6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:44:52.403138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828805fe-e54c-4114-8730-5a2ae93690f6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:44:52.408737 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05d1b7-85f4-4d99-a2e1-2f23cb591b38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d03db170fb6460794e32669456bcdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:44:52.409336 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05d1b7-85f4-4d99-a2e1-2f23cb591b38 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:52.409608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05d1b7-85f4-4d99-a2e1-2f23cb591b38 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:52.409873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05d1b7-85f4-4d99-a2e1-2f23cb591b38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:52.418476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05d1b7-85f4-4d99-a2e1-2f23cb591b38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:44:52.423739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05d1b7-85f4-4d99-a2e1-2f23cb591b38 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:44:52.430525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 768/1521] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:44:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:44:55.021220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393b980-38d5-4b0b-8067-4f87def63d59 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:55.021823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393b980-38d5-4b0b-8067-4f87def63d59 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:55.021823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393b980-38d5-4b0b-8067-4f87def63d59 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:55.023165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 08:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:44:55.029161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c8d8a-63e1-40b2-af51-17fc7e1400d5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:55.029596 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c8d8a-63e1-40b2-af51-17fc7e1400d5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:55.030296 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c8d8a-63e1-40b2-af51-17fc7e1400d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:55.049619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-699c8d8a-63e1-40b2-af51-17fc7e1400d5 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:44:55.050838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-699c8d8a-63e1-40b2-af51-17fc7e1400d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:44:55.058563 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 08:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:44:55.212818 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393b980-38d5-4b0b-8067-4f87def63d59 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:44:55.228015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393b980-38d5-4b0b-8067-4f87def63d59 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:44:55.238328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195058ae-941e-46f9-b63e-afb3b04b9b77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:44:55.238328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195058ae-941e-46f9-b63e-afb3b04b9b77 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:55.238328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195058ae-941e-46f9-b63e-afb3b04b9b77 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:55.238328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195058ae-941e-46f9-b63e-afb3b04b9b77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:55.242956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195058ae-941e-46f9-b63e-afb3b04b9b77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:44:55.244525 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195058ae-941e-46f9-b63e-afb3b04b9b77 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:44:55.254397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 755/1524] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:44:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:44:56.330671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0edb80-0ae6-4544-a816-cdf9ad20cfbb None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:56.331281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0edb80-0ae6-4544-a816-cdf9ad20cfbb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:56.331281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0edb80-0ae6-4544-a816-cdf9ad20cfbb None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:56.332573 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 08:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:44:56.338804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164830db-85c6-4c7d-b3f2-e689055cb713 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:56.339339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164830db-85c6-4c7d-b3f2-e689055cb713 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:56.339438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164830db-85c6-4c7d-b3f2-e689055cb713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:56.359945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-164830db-85c6-4c7d-b3f2-e689055cb713 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:44:56.361142 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-164830db-85c6-4c7d-b3f2-e689055cb713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:44:56.369047 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 08:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:44:56.521435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0edb80-0ae6-4544-a816-cdf9ad20cfbb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:44:56.536363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0edb80-0ae6-4544-a816-cdf9ad20cfbb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:44:56.541748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b6a6d-d657-479b-92dc-707913fd4d2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:44:56.542367 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6a6d-d657-479b-92dc-707913fd4d2b service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:44:56.542641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6a6d-d657-479b-92dc-707913fd4d2b service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:44:56.542905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6a6d-d657-479b-92dc-707913fd4d2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:44:56.551154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b6a6d-d657-479b-92dc-707913fd4d2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:44:56.552756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b6a6d-d657-479b-92dc-707913fd4d2b service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:44:56.561995 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 771/1527] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:44:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:45:00.526969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a853b85-f76a-4091-baeb-caae5bef2557 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:45:00.527550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a853b85-f76a-4091-baeb-caae5bef2557 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:45:00.527550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a853b85-f76a-4091-baeb-caae5bef2557 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:45:00.528855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:45:00.534814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1613428-8e12-4390-92f0-9fb517497e49 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:45:00.535234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1613428-8e12-4390-92f0-9fb517497e49 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:45:00.535332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1613428-8e12-4390-92f0-9fb517497e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:45:00.554463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d1613428-8e12-4390-92f0-9fb517497e49 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:45:00.555644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d1613428-8e12-4390-92f0-9fb517497e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:45:00.563505 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:45:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:45:00.852858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a853b85-f76a-4091-baeb-caae5bef2557 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:45:00.854591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7a853b85-f76a-4091-baeb-caae5bef2557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:45:00.933220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a853b85-f76a-4091-baeb-caae5bef2557 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:45:00.937267 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5da57-42b9-4ce8-9b48-d821036ca0cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:45:00.937859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5da57-42b9-4ce8-9b48-d821036ca0cb service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:45:00.938131 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5da57-42b9-4ce8-9b48-d821036ca0cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:45:00.938421 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5da57-42b9-4ce8-9b48-d821036ca0cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:45:00.947060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5da57-42b9-4ce8-9b48-d821036ca0cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:45:00.948467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5da57-42b9-4ce8-9b48-d821036ca0cb service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:45:00.958277 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 758/1530] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:45:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:45:03.117232 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4188e-2c9e-49f8-90d6-59b626683096 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:45:03.117806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4188e-2c9e-49f8-90d6-59b626683096 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:45:03.117924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4188e-2c9e-49f8-90d6-59b626683096 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:45:03.119288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:45:03.130350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a1b33-1f06-4586-87ac-8842159a48b6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:45:03.130350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a1b33-1f06-4586-87ac-8842159a48b6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:45:03.130350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a1b33-1f06-4586-87ac-8842159a48b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:45:03.146265 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-da8a1b33-1f06-4586-87ac-8842159a48b6 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:45:03.147488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-da8a1b33-1f06-4586-87ac-8842159a48b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:45:03.155939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:45:03.447466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4188e-2c9e-49f8-90d6-59b626683096 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:45:03.449256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2fb4188e-2c9e-49f8-90d6-59b626683096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:45:03.462819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4188e-2c9e-49f8-90d6-59b626683096 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:45:03.466340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401132e2-b4c9-4a98-9396-517c89cb908e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:45:03.466922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401132e2-b4c9-4a98-9396-517c89cb908e service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:45:03.467197 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401132e2-b4c9-4a98-9396-517c89cb908e service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:45:03.467466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401132e2-b4c9-4a98-9396-517c89cb908e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:45:03.476612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401132e2-b4c9-4a98-9396-517c89cb908e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:45:03.478034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401132e2-b4c9-4a98-9396-517c89cb908e service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:45:03.488288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 774/1533] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:45:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:45:07.141287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313ff16-d796-4e6e-8727-17e3b01e5b3e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:45:07.141837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313ff16-d796-4e6e-8727-17e3b01e5b3e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:45:07.141837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313ff16-d796-4e6e-8727-17e3b01e5b3e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:45:07.143105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:45:07.148439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423583a8-3dcb-451f-91ba-ef25d21b9dc4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:45:07.148923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423583a8-3dcb-451f-91ba-ef25d21b9dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:45:07.149032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423583a8-3dcb-451f-91ba-ef25d21b9dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:45:07.157293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:45:07.169250 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-423583a8-3dcb-451f-91ba-ef25d21b9dc4 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:45:07.170919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-423583a8-3dcb-451f-91ba-ef25d21b9dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:45:07.178979 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:45:07.482234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313ff16-d796-4e6e-8727-17e3b01e5b3e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:45:07.483932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1313ff16-d796-4e6e-8727-17e3b01e5b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:45:07.560994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313ff16-d796-4e6e-8727-17e3b01e5b3e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:45:07.564631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ed87c-50d0-4e35-a312-0da69635de1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:45:07.565227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ed87c-50d0-4e35-a312-0da69635de1c service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:45:07.565505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ed87c-50d0-4e35-a312-0da69635de1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:45:07.565775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ed87c-50d0-4e35-a312-0da69635de1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:45:07.575053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ed87c-50d0-4e35-a312-0da69635de1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:45:07.577034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ed87c-50d0-4e35-a312-0da69635de1c service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:45:07.586738 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 761/1536] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:45:09.744141 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadcff9-a29a-42f5-813e-e373ce7df925 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:45:09.744763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadcff9-a29a-42f5-813e-e373ce7df925 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:45:09.744763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadcff9-a29a-42f5-813e-e373ce7df925 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:45:09.746002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 08:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:45:09.752020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27f007-6e1b-4fb7-ad38-2dbab27c4717 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:45:09.752527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27f007-6e1b-4fb7-ad38-2dbab27c4717 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:45:09.752630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27f007-6e1b-4fb7-ad38-2dbab27c4717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:45:09.772571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4b27f007-6e1b-4fb7-ad38-2dbab27c4717 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:45:09.773778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4b27f007-6e1b-4fb7-ad38-2dbab27c4717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:45:09.782510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 08:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:45:10.085209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadcff9-a29a-42f5-813e-e373ce7df925 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:45:10.086902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-adadcff9-a29a-42f5-813e-e373ce7df925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:45:10.100160 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadcff9-a29a-42f5-813e-e373ce7df925 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:45:10.103797 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74553b84-13ca-486d-8ef8-d07d8b3b1bb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:45:10.104397 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74553b84-13ca-486d-8ef8-d07d8b3b1bb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:45:10.104673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74553b84-13ca-486d-8ef8-d07d8b3b1bb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:45:10.104943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74553b84-13ca-486d-8ef8-d07d8b3b1bb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:45:10.114919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74553b84-13ca-486d-8ef8-d07d8b3b1bb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:45:10.116366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74553b84-13ca-486d-8ef8-d07d8b3b1bb4 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:45:10.125865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 777/1539] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:45:19.767653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:45:20.121599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=8 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:32.680336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b267b94-449e-44f7-a2d4-fb5dd72671a2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:32.680976 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b267b94-449e-44f7-a2d4-fb5dd72671a2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:32.680976 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b267b94-449e-44f7-a2d4-fb5dd72671a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:32.713048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8b267b94-449e-44f7-a2d4-fb5dd72671a2 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:32.714308 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8b267b94-449e-44f7-a2d4-fb5dd72671a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:32.726227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 763/1540] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:46:32.736160 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0cab8-40a3-46f0-9e6f-03e4fb6ac1be None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:32.736786 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0cab8-40a3-46f0-9e6f-03e4fb6ac1be None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:32.736786 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0cab8-40a3-46f0-9e6f-03e4fb6ac1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:32.765757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-08f0cab8-40a3-46f0-9e6f-03e4fb6ac1be None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:32.767040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-08f0cab8-40a3-46f0-9e6f-03e4fb6ac1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:32.784399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 778/1541] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:32.789517 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b267b94-449e-44f7-a2d4-fb5dd72671a2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:32.804121 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b267b94-449e-44f7-a2d4-fb5dd72671a2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:32.812812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57864cab-0718-4f87-a8dd-ed57e20c0dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:32.813558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57864cab-0718-4f87-a8dd-ed57e20c0dfc admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:32.813839 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57864cab-0718-4f87-a8dd-ed57e20c0dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:32.814116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57864cab-0718-4f87-a8dd-ed57e20c0dfc admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:32.814805 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57864cab-0718-4f87-a8dd-ed57e20c0dfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:46:32.815105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57864cab-0718-4f87-a8dd-ed57e20c0dfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:32.816117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-57864cab-0718-4f87-a8dd-ed57e20c0dfc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:46:32.816117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Sep 11 08:46:32.816117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: Sep 11 08:46:32.817006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57864cab-0718-4f87-a8dd-ed57e20c0dfc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:32.840901 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 764/1542] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:46:32.849292 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0cab8-40a3-46f0-9e6f-03e4fb6ac1be None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:32.858587 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4467e-c6ef-4e51-850f-9d2bf5534af5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:32.858587 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4467e-c6ef-4e51-850f-9d2bf5534af5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:32.858587 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4467e-c6ef-4e51-850f-9d2bf5534af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:32.866171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0cab8-40a3-46f0-9e6f-03e4fb6ac1be None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:32.872091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c19e5a-5038-45cb-a3a7-1edc75311ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:32.872711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c19e5a-5038-45cb-a3a7-1edc75311ccf admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:32.872984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c19e5a-5038-45cb-a3a7-1edc75311ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:32.873252 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c19e5a-5038-45cb-a3a7-1edc75311ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:32.873894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c19e5a-5038-45cb-a3a7-1edc75311ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:32.874968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-94c19e5a-5038-45cb-a3a7-1edc75311ccf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:46:32.874968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Sep 11 08:46:32.874968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: Sep 11 08:46:32.881552 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c19e5a-5038-45cb-a3a7-1edc75311ccf admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:32.897594 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f8a4467e-c6ef-4e51-850f-9d2bf5534af5 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:32.903683 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f8a4467e-c6ef-4e51-850f-9d2bf5534af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:32.929381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 765/1543] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:32.948728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd890c5-f4b2-4bd4-b9ea-ccba748ee925 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:32.948937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd890c5-f4b2-4bd4-b9ea-ccba748ee925 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:32.949259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd890c5-f4b2-4bd4-b9ea-ccba748ee925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:32.985856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1bd890c5-f4b2-4bd4-b9ea-ccba748ee925 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:32.987109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1bd890c5-f4b2-4bd4-b9ea-ccba748ee925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:33.003791 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 766/1544] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:46:33.008489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd890c5-f4b2-4bd4-b9ea-ccba748ee925 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.021773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd890c5-f4b2-4bd4-b9ea-ccba748ee925 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.030954 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8c663-ea4f-433e-88a0-018793e0a1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.031576 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8c663-ea4f-433e-88a0-018793e0a1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.031849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8c663-ea4f-433e-88a0-018793e0a1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.032113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8c663-ea4f-433e-88a0-018793e0a1e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.032571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 779/1545] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:32 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:33.032835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8c663-ea4f-433e-88a0-018793e0a1e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:46:33.033132 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8c663-ea4f-433e-88a0-018793e0a1e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.034808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8c663-ea4f-433e-88a0-018793e0a1e3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.037459 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c19e5a-5038-45cb-a3a7-1edc75311ccf admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.053949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 767/1546] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:46:33.076617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8c663-ea4f-433e-88a0-018793e0a1e3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.100859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8c663-ea4f-433e-88a0-018793e0a1e3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.126201 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c19e5a-5038-45cb-a3a7-1edc75311ccf admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.138909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9a35b-6a4c-4226-93bf-a95cebe15254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.139540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a35b-6a4c-4226-93bf-a95cebe15254 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.139817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a35b-6a4c-4226-93bf-a95cebe15254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.140099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a35b-6a4c-4226-93bf-a95cebe15254 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.140742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9a35b-6a4c-4226-93bf-a95cebe15254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.142726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9a35b-6a4c-4226-93bf-a95cebe15254 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.214568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce448f2-a217-43eb-adb2-71e37b3b0eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.216099 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce448f2-a217-43eb-adb2-71e37b3b0eef admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.216392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce448f2-a217-43eb-adb2-71e37b3b0eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.216721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce448f2-a217-43eb-adb2-71e37b3b0eef admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.217370 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce448f2-a217-43eb-adb2-71e37b3b0eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.219338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce448f2-a217-43eb-adb2-71e37b3b0eef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.312822 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 780/1547] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/users => generated 287 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:33.314204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07ee5b-523f-448f-9138-322ed85fc58b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.314488 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07ee5b-523f-448f-9138-322ed85fc58b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.314756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07ee5b-523f-448f-9138-322ed85fc58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.336390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-dd07ee5b-523f-448f-9138-322ed85fc58b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:33.341541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dd07ee5b-523f-448f-9138-322ed85fc58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:33.345549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 768/1548] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/projects => generated 377 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:46:33.345549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67230b71-4d17-4bbd-8268-5abaf2aa330d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.345549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67230b71-4d17-4bbd-8268-5abaf2aa330d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.345549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67230b71-4d17-4bbd-8268-5abaf2aa330d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.352623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 781/1549] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:33.353560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6785323-b773-46d3-a3d3-93cf02ca676e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.353957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6785323-b773-46d3-a3d3-93cf02ca676e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.354260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6785323-b773-46d3-a3d3-93cf02ca676e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.370402 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-67230b71-4d17-4bbd-8268-5abaf2aa330d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:33.372297 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-67230b71-4d17-4bbd-8268-5abaf2aa330d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:33.378172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e6785323-b773-46d3-a3d3-93cf02ca676e None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:33.379423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e6785323-b773-46d3-a3d3-93cf02ca676e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:33.432779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 769/1550] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:33.434777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f44a49-3a7a-4fe3-bb27-7bbabb513a72 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.435011 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f44a49-3a7a-4fe3-bb27-7bbabb513a72 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.435343 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f44a49-3a7a-4fe3-bb27-7bbabb513a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.449747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 782/1551] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:33.449747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f310e-4c54-4c1d-b4dd-1e6277f0215d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.449747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f310e-4c54-4c1d-b4dd-1e6277f0215d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.449747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f310e-4c54-4c1d-b4dd-1e6277f0215d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.462807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f6f44a49-3a7a-4fe3-bb27-7bbabb513a72 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:33.463946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6f44a49-3a7a-4fe3-bb27-7bbabb513a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:33.466397 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c99f310e-4c54-4c1d-b4dd-1e6277f0215d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:33.467583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c99f310e-4c54-4c1d-b4dd-1e6277f0215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:33.472621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 770/1552] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:33.473309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f44a49-3a7a-4fe3-bb27-7bbabb513a72 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.477531 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 783/1553] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:33.478242 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f310e-4c54-4c1d-b4dd-1e6277f0215d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.488568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f310e-4c54-4c1d-b4dd-1e6277f0215d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.490753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f44a49-3a7a-4fe3-bb27-7bbabb513a72 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.493582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a926c-cc45-45b6-b08d-be55ca1d67a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.494229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a926c-cc45-45b6-b08d-be55ca1d67a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.494484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a926c-cc45-45b6-b08d-be55ca1d67a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.494754 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a926c-cc45-45b6-b08d-be55ca1d67a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.495379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a926c-cc45-45b6-b08d-be55ca1d67a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.495487 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fb54d-b2f3-4545-ab50-4a8ae4cf64a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.496574 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fb54d-b2f3-4545-ab50-4a8ae4cf64a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.496845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fb54d-b2f3-4545-ab50-4a8ae4cf64a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.497145 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fb54d-b2f3-4545-ab50-4a8ae4cf64a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.497781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fb54d-b2f3-4545-ab50-4a8ae4cf64a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.497886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a926c-cc45-45b6-b08d-be55ca1d67a9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.499394 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fb54d-b2f3-4545-ab50-4a8ae4cf64a7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.510310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 771/1554] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:33.511572 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffc6c5-024d-47a0-81dc-3690fd561162 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.512365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffc6c5-024d-47a0-81dc-3690fd561162 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.512634 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffc6c5-024d-47a0-81dc-3690fd561162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.532792 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cbffc6c5-024d-47a0-81dc-3690fd561162 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:33.533995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cbffc6c5-024d-47a0-81dc-3690fd561162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:33.541889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 772/1555] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:33.543116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381ba36-bea7-48af-b758-220440ddcc62 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.543394 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381ba36-bea7-48af-b758-220440ddcc62 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.543664 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381ba36-bea7-48af-b758-220440ddcc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.564031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1381ba36-bea7-48af-b758-220440ddcc62 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:33.565222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1381ba36-bea7-48af-b758-220440ddcc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:33.573212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 773/1556] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:33.574987 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6f00e-a4dd-4872-bccb-a86c4da79d1a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.575295 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6f00e-a4dd-4872-bccb-a86c4da79d1a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.575578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6f00e-a4dd-4872-bccb-a86c4da79d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.587597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 784/1557] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:33.587789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442069eb-3f89-46a7-bfec-21ac3f07fca4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.593436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442069eb-3f89-46a7-bfec-21ac3f07fca4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.593436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442069eb-3f89-46a7-bfec-21ac3f07fca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.594985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-15c6f00e-a4dd-4872-bccb-a86c4da79d1a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:33.596202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-15c6f00e-a4dd-4872-bccb-a86c4da79d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:33.604418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 774/1558] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:33.608428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b59c9b-8eb8-4ef6-ae59-dc08b73db496 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.608428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b59c9b-8eb8-4ef6-ae59-dc08b73db496 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.608428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b59c9b-8eb8-4ef6-ae59-dc08b73db496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.610821 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-442069eb-3f89-46a7-bfec-21ac3f07fca4 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:33.612011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-442069eb-3f89-46a7-bfec-21ac3f07fca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:33.614199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:33.619984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 785/1559] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:33.621707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb27613-766e-4f3b-88cd-bd87ca8e660c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.621986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb27613-766e-4f3b-88cd-bd87ca8e660c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.622298 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb27613-766e-4f3b-88cd-bd87ca8e660c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.627324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-64b59c9b-8eb8-4ef6-ae59-dc08b73db496 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:33.628523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-64b59c9b-8eb8-4ef6-ae59-dc08b73db496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:33.638292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 775/1560] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:33.639059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b59c9b-8eb8-4ef6-ae59-dc08b73db496 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.642803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1cb27613-766e-4f3b-88cd-bd87ca8e660c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:33.643980 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1cb27613-766e-4f3b-88cd-bd87ca8e660c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:33.649537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b59c9b-8eb8-4ef6-ae59-dc08b73db496 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.652054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 786/1561] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:33.653972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44271b-5fae-44cc-be43-35b9882a4ad6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.654287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44271b-5fae-44cc-be43-35b9882a4ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.654887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44271b-5fae-44cc-be43-35b9882a4ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.657909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8923bf0-a1df-41f8-878e-3e7133da6bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.658662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8923bf0-a1df-41f8-878e-3e7133da6bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.658942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8923bf0-a1df-41f8-878e-3e7133da6bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.659701 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8923bf0-a1df-41f8-878e-3e7133da6bc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.659844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8923bf0-a1df-41f8-878e-3e7133da6bc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.661482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8923bf0-a1df-41f8-878e-3e7133da6bc0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.674125 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 776/1562] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:33.675146 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8923bf0-a1df-41f8-878e-3e7133da6bc0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.677732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ed44271b-5fae-44cc-be43-35b9882a4ad6 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:33.677925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ed44271b-5fae-44cc-be43-35b9882a4ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:33.685929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8923bf0-a1df-41f8-878e-3e7133da6bc0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.690730 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 787/1563] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:33.690730 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44271b-5fae-44cc-be43-35b9882a4ad6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.692413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826b896-2345-43cb-80cb-6174b9e4efb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.693024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826b896-2345-43cb-80cb-6174b9e4efb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.693411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:33.693530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826b896-2345-43cb-80cb-6174b9e4efb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.693813 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826b896-2345-43cb-80cb-6174b9e4efb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.694503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826b896-2345-43cb-80cb-6174b9e4efb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:46:33.695358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826b896-2345-43cb-80cb-6174b9e4efb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.696402 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826b896-2345-43cb-80cb-6174b9e4efb0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.699910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44271b-5fae-44cc-be43-35b9882a4ad6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.705479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c744354-b35b-4a42-af78-b55d30a2a610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.706259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c744354-b35b-4a42-af78-b55d30a2a610 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.706660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c744354-b35b-4a42-af78-b55d30a2a610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.706934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c744354-b35b-4a42-af78-b55d30a2a610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.707590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c744354-b35b-4a42-af78-b55d30a2a610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.709189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c744354-b35b-4a42-af78-b55d30a2a610 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.709607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 777/1564] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:46:33.710365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826b896-2345-43cb-80cb-6174b9e4efb0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.723705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 788/1565] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:33.723705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c744354-b35b-4a42-af78-b55d30a2a610 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.723981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826b896-2345-43cb-80cb-6174b9e4efb0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.729560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741f7cd-0d4c-4b14-9466-910ec2f3d967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.730252 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f7cd-0d4c-4b14-9466-910ec2f3d967 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.731072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f7cd-0d4c-4b14-9466-910ec2f3d967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.731338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f7cd-0d4c-4b14-9466-910ec2f3d967 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.732237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741f7cd-0d4c-4b14-9466-910ec2f3d967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:46:33.732532 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741f7cd-0d4c-4b14-9466-910ec2f3d967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.733216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c744354-b35b-4a42-af78-b55d30a2a610 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.734140 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741f7cd-0d4c-4b14-9466-910ec2f3d967 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.739106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c2ea9-fa60-4ba8-bab0-597b47add940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.739704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c2ea9-fa60-4ba8-bab0-597b47add940 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.739984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c2ea9-fa60-4ba8-bab0-597b47add940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.740252 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c2ea9-fa60-4ba8-bab0-597b47add940 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.741348 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c2ea9-fa60-4ba8-bab0-597b47add940 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:46:33.741676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c2ea9-fa60-4ba8-bab0-597b47add940 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.743809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c2ea9-fa60-4ba8-bab0-597b47add940 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.753559 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 778/1566] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:46:33.753559 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741f7cd-0d4c-4b14-9466-910ec2f3d967 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.764700 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 789/1567] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:46:33.764894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741f7cd-0d4c-4b14-9466-910ec2f3d967 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.765002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c2ea9-fa60-4ba8-bab0-597b47add940 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.765103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac4735-d828-41ea-aa6c-b179fd1560d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.765103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac4735-d828-41ea-aa6c-b179fd1560d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.765103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac4735-d828-41ea-aa6c-b179fd1560d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.765103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac4735-d828-41ea-aa6c-b179fd1560d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.765420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac4735-d828-41ea-aa6c-b179fd1560d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:46:33.765420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac4735-d828-41ea-aa6c-b179fd1560d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.766647 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac4735-d828-41ea-aa6c-b179fd1560d2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.768171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c2ea9-fa60-4ba8-bab0-597b47add940 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.772754 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8945d-b8bd-421e-bee1-b774874d4594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.773346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8945d-b8bd-421e-bee1-b774874d4594 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.773619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8945d-b8bd-421e-bee1-b774874d4594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.773890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8945d-b8bd-421e-bee1-b774874d4594 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.774558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8945d-b8bd-421e-bee1-b774874d4594 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:46:33.774855 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8945d-b8bd-421e-bee1-b774874d4594 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.776436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8945d-b8bd-421e-bee1-b774874d4594 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.779373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 779/1568] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:46:33.780166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac4735-d828-41ea-aa6c-b179fd1560d2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.790650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 790/1569] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:46:33.790822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac4735-d828-41ea-aa6c-b179fd1560d2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.791531 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8945d-b8bd-421e-bee1-b774874d4594 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.795326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a2b3c-01e2-4b0d-ba55-8ff21d78e3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.795980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a2b3c-01e2-4b0d-ba55-8ff21d78e3eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.796259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a2b3c-01e2-4b0d-ba55-8ff21d78e3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.796558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a2b3c-01e2-4b0d-ba55-8ff21d78e3eb admin admin] PATH_INFO: `/v3/projects/f96c7a52c30e4043908a94348eb6af9b/users/ca42316192ec4c62b98ead2e7d659880/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.802054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8945d-b8bd-421e-bee1-b774874d4594 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.806926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c5e7a-664e-4ca3-bb41-d4a2562460f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.807516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c5e7a-664e-4ca3-bb41-d4a2562460f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.807790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c5e7a-664e-4ca3-bb41-d4a2562460f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.808064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c5e7a-664e-4ca3-bb41-d4a2562460f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.809263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c5e7a-664e-4ca3-bb41-d4a2562460f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:46:33.809568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c5e7a-664e-4ca3-bb41-d4a2562460f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.811192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c5e7a-664e-4ca3-bb41-d4a2562460f7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.812524 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a2b3c-01e2-4b0d-ba55-8ff21d78e3eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96c7a52c30e4043908a94348eb6af9b, user_id=ca42316192ec4c62b98ead2e7d659880, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.815948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a2b3c-01e2-4b0d-ba55-8ff21d78e3eb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.823401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 791/1570] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:46:33.824922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c5e7a-664e-4ca3-bb41-d4a2562460f7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.834623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c5e7a-664e-4ca3-bb41-d4a2562460f7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.839265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1e145-7626-4319-9634-11fd3ad9cc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.839925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1e145-7626-4319-9634-11fd3ad9cc3d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.840195 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1e145-7626-4319-9634-11fd3ad9cc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.840462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1e145-7626-4319-9634-11fd3ad9cc3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.841077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1e145-7626-4319-9634-11fd3ad9cc3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.843000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1e145-7626-4319-9634-11fd3ad9cc3d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.887866 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 780/1571] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:33 2022] PUT /identity/v3/projects/f96c7a52c30e4043908a94348eb6af9b/users/ca42316192ec4c62b98ead2e7d659880/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:33.888761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a2b3c-01e2-4b0d-ba55-8ff21d78e3eb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.950531 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a2b3c-01e2-4b0d-ba55-8ff21d78e3eb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:33.956222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa33c28-f508-46dd-832e-29e57e058a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:33.957156 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa33c28-f508-46dd-832e-29e57e058a68 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:33.957600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa33c28-f508-46dd-832e-29e57e058a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:33.958122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa33c28-f508-46dd-832e-29e57e058a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:33.959136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa33c28-f508-46dd-832e-29e57e058a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:33.962065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa33c28-f508-46dd-832e-29e57e058a68 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:33.970654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 792/1572] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:33.970654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1e145-7626-4319-9634-11fd3ad9cc3d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.974852 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 781/1573] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:33.975574 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa33c28-f508-46dd-832e-29e57e058a68 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:33.985908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa33c28-f508-46dd-832e-29e57e058a68 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:34.035348 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1e145-7626-4319-9634-11fd3ad9cc3d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:34.040137 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43b2ab-ea6b-493a-a1b4-8b022f2f3dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:34.040736 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43b2ab-ea6b-493a-a1b4-8b022f2f3dbe admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:34.041010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43b2ab-ea6b-493a-a1b4-8b022f2f3dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:34.041280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43b2ab-ea6b-493a-a1b4-8b022f2f3dbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:34.041919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43b2ab-ea6b-493a-a1b4-8b022f2f3dbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:34.043607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8192d-5847-46f4-b832-9cd4d3061363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:34.044206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8192d-5847-46f4-b832-9cd4d3061363 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:34.044428 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43b2ab-ea6b-493a-a1b4-8b022f2f3dbe admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:34.044526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8192d-5847-46f4-b832-9cd4d3061363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:34.044735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8192d-5847-46f4-b832-9cd4d3061363 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:34.045354 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8192d-5847-46f4-b832-9cd4d3061363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:34.047279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8192d-5847-46f4-b832-9cd4d3061363 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:34.140037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 793/1574] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/projects => generated 363 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:34.140837 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43b2ab-ea6b-493a-a1b4-8b022f2f3dbe admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:34.153972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 782/1575] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:33 2022] POST /identity/v3/projects => generated 403 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:34.155079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8192d-5847-46f4-b832-9cd4d3061363 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:34.159337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:34.202574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43b2ab-ea6b-493a-a1b4-8b022f2f3dbe admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:34.219994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8192d-5847-46f4-b832-9cd4d3061363 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:34.221889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:34.225513 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827cdcbb-f30c-4b6c-96b4-183d180c0d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:34.226057 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cdcbb-f30c-4b6c-96b4-183d180c0d2c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:34.226362 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cdcbb-f30c-4b6c-96b4-183d180c0d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:34.226631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cdcbb-f30c-4b6c-96b4-183d180c0d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:34.227328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827cdcbb-f30c-4b6c-96b4-183d180c0d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:34.229236 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827cdcbb-f30c-4b6c-96b4-183d180c0d2c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:34.261612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68236f-d06e-438d-b4dc-e38fb544a813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:34.262248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68236f-d06e-438d-b4dc-e38fb544a813 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:34.262520 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68236f-d06e-438d-b4dc-e38fb544a813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:34.262790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68236f-d06e-438d-b4dc-e38fb544a813 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:34.263412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68236f-d06e-438d-b4dc-e38fb544a813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:34.265473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68236f-d06e-438d-b4dc-e38fb544a813 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:34.317789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 783/1576] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:34 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:34.318801 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827cdcbb-f30c-4b6c-96b4-183d180c0d2c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:34.380435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827cdcbb-f30c-4b6c-96b4-183d180c0d2c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:34.384983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa65872-3f28-4dde-92a3-19c120b3346a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:34.390255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa65872-3f28-4dde-92a3-19c120b3346a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:34.390255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa65872-3f28-4dde-92a3-19c120b3346a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:34.390255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa65872-3f28-4dde-92a3-19c120b3346a admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:34.390255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa65872-3f28-4dde-92a3-19c120b3346a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:34.390255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa65872-3f28-4dde-92a3-19c120b3346a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:34.396938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 794/1577] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:34 2022] POST /identity/v3/projects => generated 375 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:34.396938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29893c-5c7c-4631-84b8-35f389fc4fd3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:34.396938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29893c-5c7c-4631-84b8-35f389fc4fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:34.396938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29893c-5c7c-4631-84b8-35f389fc4fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:34.424696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ce29893c-5c7c-4631-84b8-35f389fc4fd3 None None] MFA Rules not processed for user `ca42316192ec4c62b98ead2e7d659880`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:34.426062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ce29893c-5c7c-4631-84b8-35f389fc4fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:34.491007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 795/1578] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:34.491763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29893c-5c7c-4631-84b8-35f389fc4fd3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:34.496669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 784/1579] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:34 2022] POST /identity/v3/projects => generated 393 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:34.497683 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa65872-3f28-4dde-92a3-19c120b3346a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:34.565653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa65872-3f28-4dde-92a3-19c120b3346a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:34.568904 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29893c-5c7c-4631-84b8-35f389fc4fd3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:34.571510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe41e0-ada6-436a-a68b-53f3021c31f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:34.572179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe41e0-ada6-436a-a68b-53f3021c31f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:34.572458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe41e0-ada6-436a-a68b-53f3021c31f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:34.572728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe41e0-ada6-436a-a68b-53f3021c31f6 admin admin] PATH_INFO: `/v3/projects/7df041c538da419d8576d746b392c4db/users/00407384465c41848ad28a15729a95d6/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:34.573571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa83a7-4f7e-4e8c-89d3-4612b553c9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:34.574770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa83a7-4f7e-4e8c-89d3-4612b553c9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:34.575041 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa83a7-4f7e-4e8c-89d3-4612b553c9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:34.577708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa83a7-4f7e-4e8c-89d3-4612b553c9c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:34.577708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa83a7-4f7e-4e8c-89d3-4612b553c9c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:34.577931 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa83a7-4f7e-4e8c-89d3-4612b553c9c3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:34.589542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe41e0-ada6-436a-a68b-53f3021c31f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df041c538da419d8576d746b392c4db, user_id=00407384465c41848ad28a15729a95d6, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:34.592303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe41e0-ada6-436a-a68b-53f3021c31f6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:34.595436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:34.678454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 796/1580] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:34 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:34.679276 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa83a7-4f7e-4e8c-89d3-4612b553c9c3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:34.690949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa83a7-4f7e-4e8c-89d3-4612b553c9c3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:34.695108 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 785/1581] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:34 2022] PUT /identity/v3/projects/7df041c538da419d8576d746b392c4db/users/00407384465c41848ad28a15729a95d6/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:34.695988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe41e0-ada6-436a-a68b-53f3021c31f6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:34.721064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:34.757661 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0557bc-108e-4a20-bce5-b680c2dd0fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:34.758394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0557bc-108e-4a20-bce5-b680c2dd0fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:34.758495 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0557bc-108e-4a20-bce5-b680c2dd0fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:34.758744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0557bc-108e-4a20-bce5-b680c2dd0fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:34.759382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0557bc-108e-4a20-bce5-b680c2dd0fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:34.761326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0557bc-108e-4a20-bce5-b680c2dd0fc1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:34.769802 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe41e0-ada6-436a-a68b-53f3021c31f6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:34.775740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a01a7a-b933-409a-b6ee-0aaa16183ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:34.776350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a01a7a-b933-409a-b6ee-0aaa16183ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:34.776630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a01a7a-b933-409a-b6ee-0aaa16183ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:34.776903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a01a7a-b933-409a-b6ee-0aaa16183ce4 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:34.778173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a01a7a-b933-409a-b6ee-0aaa16183ce4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:34.783658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-79a01a7a-b933-409a-b6ee-0aaa16183ce4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:46:34.783658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Sep 11 08:46:34.783658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: Sep 11 08:46:34.783658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a01a7a-b933-409a-b6ee-0aaa16183ce4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:34.901120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 797/1582] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:34 2022] POST /identity/v3/users => generated 282 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:34.901396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 786/1583] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:34 2022] POST /identity/v3/users => generated 302 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:34.902865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a01a7a-b933-409a-b6ee-0aaa16183ce4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:34.905997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0557bc-108e-4a20-bce5-b680c2dd0fc1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:34.913896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a01a7a-b933-409a-b6ee-0aaa16183ce4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:34.916475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0557bc-108e-4a20-bce5-b680c2dd0fc1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:34.919813 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5e803-12c7-4e6f-ad09-f3688e3ccdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:34.920466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5e803-12c7-4e6f-ad09-f3688e3ccdda admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:34.920742 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5e803-12c7-4e6f-ad09-f3688e3ccdda admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:34.921012 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5e803-12c7-4e6f-ad09-f3688e3ccdda admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:34.921701 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5e803-12c7-4e6f-ad09-f3688e3ccdda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:34.922825 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105c9d7-a9ff-4167-8b80-6bef45a7fa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:34.927154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105c9d7-a9ff-4167-8b80-6bef45a7fa05 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:34.927302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5e803-12c7-4e6f-ad09-f3688e3ccdda admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:34.927410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105c9d7-a9ff-4167-8b80-6bef45a7fa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:34.927410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105c9d7-a9ff-4167-8b80-6bef45a7fa05 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:34.927410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105c9d7-a9ff-4167-8b80-6bef45a7fa05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:34.927410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105c9d7-a9ff-4167-8b80-6bef45a7fa05 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.038409 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 787/1584] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:34 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:35.039129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5e803-12c7-4e6f-ad09-f3688e3ccdda admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.049673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5e803-12c7-4e6f-ad09-f3688e3ccdda admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.054264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81040a69-da1a-4735-94e4-611d5acb8afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.054870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81040a69-da1a-4735-94e4-611d5acb8afe admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.055150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81040a69-da1a-4735-94e4-611d5acb8afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.055417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81040a69-da1a-4735-94e4-611d5acb8afe admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.056044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81040a69-da1a-4735-94e4-611d5acb8afe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.057960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81040a69-da1a-4735-94e4-611d5acb8afe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.067498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 798/1585] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:34 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:35.075204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074e835-8447-43ae-834e-861fc55b56cc None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.075204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074e835-8447-43ae-834e-861fc55b56cc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.075204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074e835-8447-43ae-834e-861fc55b56cc None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.075204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 799/1586] 10.222.0.44 () {54 vars in 962 bytes} [Sun Sep 11 08:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:46:35.075204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074e835-8447-43ae-834e-861fc55b56cc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.086031 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074e835-8447-43ae-834e-861fc55b56cc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.091087 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072245e4-1bba-4cf9-a090-ede04be7c02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.091713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072245e4-1bba-4cf9-a090-ede04be7c02f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.091988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072245e4-1bba-4cf9-a090-ede04be7c02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.092253 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072245e4-1bba-4cf9-a090-ede04be7c02f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.092899 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072245e4-1bba-4cf9-a090-ede04be7c02f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.094538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072245e4-1bba-4cf9-a090-ede04be7c02f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.104791 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 800/1587] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:35.106733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a41dc6-1b78-489c-8dd6-4c745f9891dd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.107267 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a41dc6-1b78-489c-8dd6-4c745f9891dd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.107536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a41dc6-1b78-489c-8dd6-4c745f9891dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.126889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-17a41dc6-1b78-489c-8dd6-4c745f9891dd None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:35.128120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-17a41dc6-1b78-489c-8dd6-4c745f9891dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:35.135780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 801/1588] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:35.137173 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a41dc6-1b78-489c-8dd6-4c745f9891dd None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.147674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a41dc6-1b78-489c-8dd6-4c745f9891dd None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.152587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe2eb9-38e5-4942-a17a-2e6871672c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.153231 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe2eb9-38e5-4942-a17a-2e6871672c2e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.153512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe2eb9-38e5-4942-a17a-2e6871672c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.153785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe2eb9-38e5-4942-a17a-2e6871672c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.154466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe2eb9-38e5-4942-a17a-2e6871672c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.156084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe2eb9-38e5-4942-a17a-2e6871672c2e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.158615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 788/1589] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:35 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:35.159488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81040a69-da1a-4735-94e4-611d5acb8afe admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.169837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81040a69-da1a-4735-94e4-611d5acb8afe admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.171539 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 802/1590] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:35.172219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe2eb9-38e5-4942-a17a-2e6871672c2e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.177550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb34ad2-dc80-4921-9f38-04e221e321e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.178185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb34ad2-dc80-4921-9f38-04e221e321e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.178711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb34ad2-dc80-4921-9f38-04e221e321e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.178992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb34ad2-dc80-4921-9f38-04e221e321e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.179643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb34ad2-dc80-4921-9f38-04e221e321e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.181299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb34ad2-dc80-4921-9f38-04e221e321e2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.182449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe2eb9-38e5-4942-a17a-2e6871672c2e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.187667 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbf13d-2d84-41a6-ab9d-46d03359496c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.188262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbf13d-2d84-41a6-ab9d-46d03359496c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.188540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbf13d-2d84-41a6-ab9d-46d03359496c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.188806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbf13d-2d84-41a6-ab9d-46d03359496c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.189442 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbf13d-2d84-41a6-ab9d-46d03359496c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.191087 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbf13d-2d84-41a6-ab9d-46d03359496c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.192928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 789/1591] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:35.196198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257fff5-daa6-4cbd-95b7-b4afea0c8cf8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.196198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257fff5-daa6-4cbd-95b7-b4afea0c8cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.196198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257fff5-daa6-4cbd-95b7-b4afea0c8cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.202673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 803/1592] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:35.203665 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbf13d-2d84-41a6-ab9d-46d03359496c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.213526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbf13d-2d84-41a6-ab9d-46d03359496c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.218945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b575bbe-05e1-469c-9721-313d18904a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.219560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b575bbe-05e1-469c-9721-313d18904a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.219839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b575bbe-05e1-469c-9721-313d18904a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.220108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b575bbe-05e1-469c-9721-313d18904a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.220762 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b575bbe-05e1-469c-9721-313d18904a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.222415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b575bbe-05e1-469c-9721-313d18904a29 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.234603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 804/1593] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:35.235434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b575bbe-05e1-469c-9721-313d18904a29 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.238914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9257fff5-daa6-4cbd-95b7-b4afea0c8cf8 None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:35.240189 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9257fff5-daa6-4cbd-95b7-b4afea0c8cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:35.246608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b575bbe-05e1-469c-9721-313d18904a29 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.251160 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562ad00-84ba-4279-9a63-2b00ae58fa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.251751 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562ad00-84ba-4279-9a63-2b00ae58fa0a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.252021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562ad00-84ba-4279-9a63-2b00ae58fa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.252282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562ad00-84ba-4279-9a63-2b00ae58fa0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.252904 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562ad00-84ba-4279-9a63-2b00ae58fa0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.254649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562ad00-84ba-4279-9a63-2b00ae58fa0a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.265376 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 805/1594] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:35.266309 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562ad00-84ba-4279-9a63-2b00ae58fa0a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.313774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 790/1595] 10.222.0.44 () {58 vars in 1059 bytes} [Sun Sep 11 08:46:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:35.314707 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257fff5-daa6-4cbd-95b7-b4afea0c8cf8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.330096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562ad00-84ba-4279-9a63-2b00ae58fa0a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.333820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ee481-102f-4044-9564-9ee26d612433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.334464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ee481-102f-4044-9564-9ee26d612433 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.334750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ee481-102f-4044-9564-9ee26d612433 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.335024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ee481-102f-4044-9564-9ee26d612433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.337596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257fff5-daa6-4cbd-95b7-b4afea0c8cf8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.342109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213d54c-af2a-4480-93cc-8ba8bbf816a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.342741 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213d54c-af2a-4480-93cc-8ba8bbf816a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.343048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213d54c-af2a-4480-93cc-8ba8bbf816a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.343319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213d54c-af2a-4480-93cc-8ba8bbf816a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.343961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213d54c-af2a-4480-93cc-8ba8bbf816a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.345555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213d54c-af2a-4480-93cc-8ba8bbf816a9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.348366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ee481-102f-4044-9564-9ee26d612433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.350718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ee481-102f-4044-9564-9ee26d612433 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.356564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 791/1596] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:35.357747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213d54c-af2a-4480-93cc-8ba8bbf816a9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.365900 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 806/1597] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:35.367085 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ee481-102f-4044-9564-9ee26d612433 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.367383 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213d54c-af2a-4480-93cc-8ba8bbf816a9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.372607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be091f6-7e3a-4d0f-bdb0-20541a63c691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.373222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be091f6-7e3a-4d0f-bdb0-20541a63c691 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.373491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be091f6-7e3a-4d0f-bdb0-20541a63c691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.373761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be091f6-7e3a-4d0f-bdb0-20541a63c691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.374434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be091f6-7e3a-4d0f-bdb0-20541a63c691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.376024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be091f6-7e3a-4d0f-bdb0-20541a63c691 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.377685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:35.380079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ee481-102f-4044-9564-9ee26d612433 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.384932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a54b5d-9c06-4843-a8c6-14ecc6af9af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.385545 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a54b5d-9c06-4843-a8c6-14ecc6af9af8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.385924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a54b5d-9c06-4843-a8c6-14ecc6af9af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.393479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a54b5d-9c06-4843-a8c6-14ecc6af9af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.393479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a54b5d-9c06-4843-a8c6-14ecc6af9af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.393479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a54b5d-9c06-4843-a8c6-14ecc6af9af8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.393761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 792/1598] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:35.393761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be091f6-7e3a-4d0f-bdb0-20541a63c691 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.399333 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 807/1599] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:35.400410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a54b5d-9c06-4843-a8c6-14ecc6af9af8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.406822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be091f6-7e3a-4d0f-bdb0-20541a63c691 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.410385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a54b5d-9c06-4843-a8c6-14ecc6af9af8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.411716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a691cb-5329-427d-8030-6b2d4f6fb978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.412459 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a691cb-5329-427d-8030-6b2d4f6fb978 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.412756 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a691cb-5329-427d-8030-6b2d4f6fb978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.413035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a691cb-5329-427d-8030-6b2d4f6fb978 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.413723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a691cb-5329-427d-8030-6b2d4f6fb978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.415441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8234b3-13be-4b51-b12e-20d28c22756e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.418738 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a691cb-5329-427d-8030-6b2d4f6fb978 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.418871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8234b3-13be-4b51-b12e-20d28c22756e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.418871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8234b3-13be-4b51-b12e-20d28c22756e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.418871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8234b3-13be-4b51-b12e-20d28c22756e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.418871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8234b3-13be-4b51-b12e-20d28c22756e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.419704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8234b3-13be-4b51-b12e-20d28c22756e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.428807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 793/1600] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:35.429623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a691cb-5329-427d-8030-6b2d4f6fb978 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.430445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 808/1601] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:35.431264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8234b3-13be-4b51-b12e-20d28c22756e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.441823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a691cb-5329-427d-8030-6b2d4f6fb978 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.444230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8234b3-13be-4b51-b12e-20d28c22756e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.447529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0843f0-3f81-4186-9bc0-184306d2a915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.447529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0843f0-3f81-4186-9bc0-184306d2a915 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.447835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0843f0-3f81-4186-9bc0-184306d2a915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.448103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0843f0-3f81-4186-9bc0-184306d2a915 admin admin] PATH_INFO: `/v3/projects/0db8bf71fdf848d3acbbd165481f143b/users/c0013ae7efed4f35a66a27e4ef17f290/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.450588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f9831-056a-4885-891e-8a826e88bff3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.451189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f9831-056a-4885-891e-8a826e88bff3 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.451464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f9831-056a-4885-891e-8a826e88bff3 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.451732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f9831-056a-4885-891e-8a826e88bff3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.464558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0843f0-3f81-4186-9bc0-184306d2a915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0db8bf71fdf848d3acbbd165481f143b, user_id=c0013ae7efed4f35a66a27e4ef17f290, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.466379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:35.468867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0843f0-3f81-4186-9bc0-184306d2a915 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.513941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f9831-056a-4885-891e-8a826e88bff3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.515582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f9831-056a-4885-891e-8a826e88bff3 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.524815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 809/1602] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:35.525654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f9831-056a-4885-891e-8a826e88bff3 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.536890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f9831-056a-4885-891e-8a826e88bff3 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.541499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd4b1b-bf22-401f-9df5-25de3e86579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.542103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd4b1b-bf22-401f-9df5-25de3e86579e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.542408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd4b1b-bf22-401f-9df5-25de3e86579e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.542749 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd4b1b-bf22-401f-9df5-25de3e86579e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.543885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd4b1b-bf22-401f-9df5-25de3e86579e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.544963 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd4b1b-bf22-401f-9df5-25de3e86579e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.556023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 810/1603] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:35.556664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd4b1b-bf22-401f-9df5-25de3e86579e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.572478 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd4b1b-bf22-401f-9df5-25de3e86579e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.572693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 794/1604] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:35 2022] PUT /identity/v3/projects/0db8bf71fdf848d3acbbd165481f143b/users/c0013ae7efed4f35a66a27e4ef17f290/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:35.573077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0843f0-3f81-4186-9bc0-184306d2a915 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.641493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ff087-fda9-4714-86ed-8646754c75c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.642542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ff087-fda9-4714-86ed-8646754c75c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.642804 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ff087-fda9-4714-86ed-8646754c75c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.643169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ff087-fda9-4714-86ed-8646754c75c7 admin admin] PATH_INFO: `/v3/projects/842fcece97a7440faab41cb09e1eb316/users/d2d0b3a45c3049c58c1b7e5e91f8e2f4/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.650344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0843f0-3f81-4186-9bc0-184306d2a915 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.656433 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c833ad-89a0-41a7-ad74-80471e067c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.657101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c833ad-89a0-41a7-ad74-80471e067c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.657378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c833ad-89a0-41a7-ad74-80471e067c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.657824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c833ad-89a0-41a7-ad74-80471e067c80 admin admin] PATH_INFO: `/v3/projects/aa8606e30b454f6d948af7cbcb809f34/users/1a0d8dda074b4f1da9f6c12dffa9b169/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.661392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ff087-fda9-4714-86ed-8646754c75c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=842fcece97a7440faab41cb09e1eb316, user_id=d2d0b3a45c3049c58c1b7e5e91f8e2f4, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.663119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-9a0ff087-fda9-4714-86ed-8646754c75c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:46:35.663119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Sep 11 08:46:35.663119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: Sep 11 08:46:35.665617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ff087-fda9-4714-86ed-8646754c75c7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.675584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c833ad-89a0-41a7-ad74-80471e067c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8606e30b454f6d948af7cbcb809f34, user_id=1a0d8dda074b4f1da9f6c12dffa9b169, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.678654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c833ad-89a0-41a7-ad74-80471e067c80 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.765497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 811/1605] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:35 2022] PUT /identity/v3/projects/842fcece97a7440faab41cb09e1eb316/users/d2d0b3a45c3049c58c1b7e5e91f8e2f4/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:35.766416 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ff087-fda9-4714-86ed-8646754c75c7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.793914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 795/1606] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:35 2022] PUT /identity/v3/projects/aa8606e30b454f6d948af7cbcb809f34/users/1a0d8dda074b4f1da9f6c12dffa9b169/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:35.796219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c833ad-89a0-41a7-ad74-80471e067c80 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:35.837263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ff087-fda9-4714-86ed-8646754c75c7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.874232 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:35.875920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c833ad-89a0-41a7-ad74-80471e067c80 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:35.880737 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1afd47-2d89-435b-a853-588a6d89a953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.881234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1afd47-2d89-435b-a853-588a6d89a953 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.881504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1afd47-2d89-435b-a853-588a6d89a953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.881721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1afd47-2d89-435b-a853-588a6d89a953 admin admin] PATH_INFO: `/v3/projects/018b87eeeda248e4a85f587c31e297f4/users/64ea6fbc6e7d42dabdf459e8de4c3621/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.899993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1afd47-2d89-435b-a853-588a6d89a953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=018b87eeeda248e4a85f587c31e297f4, user_id=64ea6fbc6e7d42dabdf459e8de4c3621, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.902840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1afd47-2d89-435b-a853-588a6d89a953 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:35.915443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d3600-9c5f-4a2e-a556-0c5141cee345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:35.916116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d3600-9c5f-4a2e-a556-0c5141cee345 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:35.916480 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d3600-9c5f-4a2e-a556-0c5141cee345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:35.916759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d3600-9c5f-4a2e-a556-0c5141cee345 admin admin] PATH_INFO: `/v3/projects/347504c603af4937b829f24349b013c2/users/3abba619decf448db477533c663004bb/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:35.931711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d3600-9c5f-4a2e-a556-0c5141cee345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=347504c603af4937b829f24349b013c2, user_id=3abba619decf448db477533c663004bb, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:35.935039 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d3600-9c5f-4a2e-a556-0c5141cee345 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:36.002710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 796/1607] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:35 2022] PUT /identity/v3/projects/018b87eeeda248e4a85f587c31e297f4/users/64ea6fbc6e7d42dabdf459e8de4c3621/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:36.003581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1afd47-2d89-435b-a853-588a6d89a953 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:36.041818 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:36.055081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 812/1608] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:35 2022] PUT /identity/v3/projects/347504c603af4937b829f24349b013c2/users/3abba619decf448db477533c663004bb/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:36.056573 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aabcf-ae28-489e-8dc5-dbba20c036da None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.056987 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aabcf-ae28-489e-8dc5-dbba20c036da None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.057411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5aabcf-ae28-489e-8dc5-dbba20c036da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.081850 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1afd47-2d89-435b-a853-588a6d89a953 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:36.089269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ea5aabcf-ae28-489e-8dc5-dbba20c036da None None] MFA Rules not processed for user `c0013ae7efed4f35a66a27e4ef17f290`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:36.091072 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ea5aabcf-ae28-489e-8dc5-dbba20c036da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:36.141446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15758aee-a4b7-4f43-bac9-c3fc4c802ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:36.142497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15758aee-a4b7-4f43-bac9-c3fc4c802ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.142871 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15758aee-a4b7-4f43-bac9-c3fc4c802ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.143167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15758aee-a4b7-4f43-bac9-c3fc4c802ed0 admin admin] PATH_INFO: `/v3/projects/bcb554b6cf504ca9a30a1ea871364e3d/users/2aea4a0e0ec74c538735b890fb774b14/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.159695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15758aee-a4b7-4f43-bac9-c3fc4c802ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb554b6cf504ca9a30a1ea871364e3d, user_id=2aea4a0e0ec74c538735b890fb774b14, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:36.161451 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 813/1609] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:36.162826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc283f0a-7a0d-49b1-b6ef-3582eb9af885 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.163100 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc283f0a-7a0d-49b1-b6ef-3582eb9af885 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.165960 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc283f0a-7a0d-49b1-b6ef-3582eb9af885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.166082 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15758aee-a4b7-4f43-bac9-c3fc4c802ed0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:36.203090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bc283f0a-7a0d-49b1-b6ef-3582eb9af885 None None] MFA Rules not processed for user `d2d0b3a45c3049c58c1b7e5e91f8e2f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:36.204372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bc283f0a-7a0d-49b1-b6ef-3582eb9af885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:36.239651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 797/1610] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:36 2022] PUT /identity/v3/projects/bcb554b6cf504ca9a30a1ea871364e3d/users/2aea4a0e0ec74c538735b890fb774b14/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:36.241146 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4559ff8-7fdc-4c88-9ad8-d140913933b7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.241485 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4559ff8-7fdc-4c88-9ad8-d140913933b7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.243191 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4559ff8-7fdc-4c88-9ad8-d140913933b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.282852 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e4559ff8-7fdc-4c88-9ad8-d140913933b7 None None] MFA Rules not processed for user `1a0d8dda074b4f1da9f6c12dffa9b169`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:36.284441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4559ff8-7fdc-4c88-9ad8-d140913933b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:36.293168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 814/1611] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:36.293168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc263e-772a-4d29-a981-ce400b2527ce None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.293168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc263e-772a-4d29-a981-ce400b2527ce None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.293168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc263e-772a-4d29-a981-ce400b2527ce None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.294686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 815/1612] 10.222.0.44 () {54 vars in 912 bytes} [Sun Sep 11 08:46:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:46:36.295845 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd767872-1101-4430-8984-5bc16c6ec54a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.296191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd767872-1101-4430-8984-5bc16c6ec54a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.296462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd767872-1101-4430-8984-5bc16c6ec54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.320531 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bd767872-1101-4430-8984-5bc16c6ec54a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:36.321808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bd767872-1101-4430-8984-5bc16c6ec54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:36.364220 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:36.370395 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 798/1613] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:36.372312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e774ae-1769-4313-a04d-f8b63f9129c8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.372601 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e774ae-1769-4313-a04d-f8b63f9129c8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.372876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e774ae-1769-4313-a04d-f8b63f9129c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.396670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e8e774ae-1769-4313-a04d-f8b63f9129c8 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:36.397940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e8e774ae-1769-4313-a04d-f8b63f9129c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:36.403818 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 816/1614] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:36.405718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faac4c-be18-4309-82bb-b3dbd7914a6a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.405718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faac4c-be18-4309-82bb-b3dbd7914a6a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.405921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faac4c-be18-4309-82bb-b3dbd7914a6a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.407259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 817/1615] 10.222.0.44 () {54 vars in 962 bytes} [Sun Sep 11 08:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:46:36.410781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 799/1616] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:36.412356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e774ae-1769-4313-a04d-f8b63f9129c8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:36.413229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9556f73-bb44-46e4-b0f4-bcf49787610d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.426704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e774ae-1769-4313-a04d-f8b63f9129c8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:36.428099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9556f73-bb44-46e4-b0f4-bcf49787610d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.429513 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9556f73-bb44-46e4-b0f4-bcf49787610d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.435401 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390be786-9f40-484b-a973-c63e0b4cd9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:36.436368 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390be786-9f40-484b-a973-c63e0b4cd9ab admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.436893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390be786-9f40-484b-a973-c63e0b4cd9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.437366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390be786-9f40-484b-a973-c63e0b4cd9ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.438560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390be786-9f40-484b-a973-c63e0b4cd9ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:36.442297 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390be786-9f40-484b-a973-c63e0b4cd9ab admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:36.487340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a9556f73-bb44-46e4-b0f4-bcf49787610d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:36.488800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a9556f73-bb44-46e4-b0f4-bcf49787610d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:36.501390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 818/1617] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:36.501390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba20f5-7ea0-47e5-a717-6634079fd7ab None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.501390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba20f5-7ea0-47e5-a717-6634079fd7ab None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.501390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba20f5-7ea0-47e5-a717-6634079fd7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.523909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-baba20f5-7ea0-47e5-a717-6634079fd7ab None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:36.525383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-baba20f5-7ea0-47e5-a717-6634079fd7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:36.541541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 800/1618] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:36 2022] POST /identity/v3/projects => generated 403 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:36.542209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390be786-9f40-484b-a973-c63e0b4cd9ab admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:36.566557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:36.614071 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 819/1619] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:36.615545 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1f6f9-6b86-4263-af06-0953690f11a5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.615768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1f6f9-6b86-4263-af06-0953690f11a5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.616049 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1f6f9-6b86-4263-af06-0953690f11a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.617553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390be786-9f40-484b-a973-c63e0b4cd9ab admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:36.623331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2350050-9652-4d84-9ae1-c19f66a434d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:36.623527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2350050-9652-4d84-9ae1-c19f66a434d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.624029 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2350050-9652-4d84-9ae1-c19f66a434d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.624452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2350050-9652-4d84-9ae1-c19f66a434d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.637893 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f7c1f6f9-6b86-4263-af06-0953690f11a5 None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:36.639137 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f7c1f6f9-6b86-4263-af06-0953690f11a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:36.689913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2350050-9652-4d84-9ae1-c19f66a434d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:36.691430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2350050-9652-4d84-9ae1-c19f66a434d9 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:36.701271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 801/1620] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:36.703833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2350050-9652-4d84-9ae1-c19f66a434d9 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:36.725147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 820/1621] 10.222.0.44 () {58 vars in 1059 bytes} [Sun Sep 11 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:36.749434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1f6f9-6b86-4263-af06-0953690f11a5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:36.797734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2350050-9652-4d84-9ae1-c19f66a434d9 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:36.801819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a11878-5501-4b6b-94fd-510ac9eef3e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:36.802514 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a11878-5501-4b6b-94fd-510ac9eef3e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.802889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a11878-5501-4b6b-94fd-510ac9eef3e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.803255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a11878-5501-4b6b-94fd-510ac9eef3e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.846788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1f6f9-6b86-4263-af06-0953690f11a5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:36.858623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7564083-68fa-4d37-85fd-48b826909d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:36.863956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7564083-68fa-4d37-85fd-48b826909d92 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.864635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7564083-68fa-4d37-85fd-48b826909d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.865090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7564083-68fa-4d37-85fd-48b826909d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.881548 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:36.902364 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a11878-5501-4b6b-94fd-510ac9eef3e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:36.904482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a11878-5501-4b6b-94fd-510ac9eef3e5 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:36.919256 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 802/1622] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:36.920103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a11878-5501-4b6b-94fd-510ac9eef3e5 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:36.930557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a11878-5501-4b6b-94fd-510ac9eef3e5 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:36.936513 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044ba42-b08e-4321-ba4a-b8bf6611e4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:36.937460 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044ba42-b08e-4321-ba4a-b8bf6611e4bd service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:36.937950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044ba42-b08e-4321-ba4a-b8bf6611e4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:36.938841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044ba42-b08e-4321-ba4a-b8bf6611e4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:36.951101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044ba42-b08e-4321-ba4a-b8bf6611e4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:36.953343 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044ba42-b08e-4321-ba4a-b8bf6611e4bd service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:36.969266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7564083-68fa-4d37-85fd-48b826909d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:36.970761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7564083-68fa-4d37-85fd-48b826909d92 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:36.971814 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 803/1623] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:36.972878 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044ba42-b08e-4321-ba4a-b8bf6611e4bd service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:36.995254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044ba42-b08e-4321-ba4a-b8bf6611e4bd service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:36.997880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 821/1624] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:36.999842 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7564083-68fa-4d37-85fd-48b826909d92 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:37.008195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fd231-d325-47bd-893b-f6b41b3ac0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:37.008819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fd231-d325-47bd-893b-f6b41b3ac0db admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:37.009249 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fd231-d325-47bd-893b-f6b41b3ac0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:37.009527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fd231-d325-47bd-893b-f6b41b3ac0db admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:37.010192 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fd231-d325-47bd-893b-f6b41b3ac0db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:37.013741 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fd231-d325-47bd-893b-f6b41b3ac0db admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:37.024660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7564083-68fa-4d37-85fd-48b826909d92 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:37.039742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c3814-3f26-4587-9f75-1f2f8ee9740f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:37.040863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c3814-3f26-4587-9f75-1f2f8ee9740f service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:37.041367 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c3814-3f26-4587-9f75-1f2f8ee9740f service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:37.041714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c3814-3f26-4587-9f75-1f2f8ee9740f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:37.045408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:37.063762 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c3814-3f26-4587-9f75-1f2f8ee9740f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:37.065277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c3814-3f26-4587-9f75-1f2f8ee9740f service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:37.081411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 822/1625] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:37.083936 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c3814-3f26-4587-9f75-1f2f8ee9740f service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:37.096177 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c3814-3f26-4587-9f75-1f2f8ee9740f service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:37.102578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74448dc-ae0a-4c14-8cb4-e02755d08e4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:37.103254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74448dc-ae0a-4c14-8cb4-e02755d08e4a service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:37.103551 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74448dc-ae0a-4c14-8cb4-e02755d08e4a service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:37.103799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74448dc-ae0a-4c14-8cb4-e02755d08e4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:37.146115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 804/1626] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:36 2022] POST /identity/v3/users => generated 308 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:37.150039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fd231-d325-47bd-893b-f6b41b3ac0db admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:37.170595 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fd231-d325-47bd-893b-f6b41b3ac0db admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:37.190462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6928efe-921a-4d65-842a-fe478071a1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:37.194902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6928efe-921a-4d65-842a-fe478071a1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:37.197292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6928efe-921a-4d65-842a-fe478071a1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:37.197811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6928efe-921a-4d65-842a-fe478071a1b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:37.202334 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6928efe-921a-4d65-842a-fe478071a1b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:37.205372 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6928efe-921a-4d65-842a-fe478071a1b6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:37.223084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 805/1627] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:46:37.227063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6928efe-921a-4d65-842a-fe478071a1b6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:37.227551 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74448dc-ae0a-4c14-8cb4-e02755d08e4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:37.229314 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74448dc-ae0a-4c14-8cb4-e02755d08e4a service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:37.241101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 823/1628] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:37.241775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6928efe-921a-4d65-842a-fe478071a1b6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:37.251093 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aabc47-564c-4d72-bd26-d4e88a3b8c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:37.252119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aabc47-564c-4d72-bd26-d4e88a3b8c39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:37.253109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aabc47-564c-4d72-bd26-d4e88a3b8c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:37.253440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aabc47-564c-4d72-bd26-d4e88a3b8c39 admin admin] PATH_INFO: `/v3/projects/8a7621566d3d4db1ae6d359ef4ee3f0a/users/be8b2c792dbc4f3d9d04583e19529570/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:37.277245 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aabc47-564c-4d72-bd26-d4e88a3b8c39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7621566d3d4db1ae6d359ef4ee3f0a, user_id=be8b2c792dbc4f3d9d04583e19529570, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:37.280421 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aabc47-564c-4d72-bd26-d4e88a3b8c39 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:37.369262 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 806/1629] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:37 2022] PUT /identity/v3/projects/8a7621566d3d4db1ae6d359ef4ee3f0a/users/be8b2c792dbc4f3d9d04583e19529570/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:46:37.374024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74448dc-ae0a-4c14-8cb4-e02755d08e4a service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:37.386496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74448dc-ae0a-4c14-8cb4-e02755d08e4a service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:37.466449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8511b-5f96-4304-846d-b2b96690bef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:37.467436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8511b-5f96-4304-846d-b2b96690bef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:37.467824 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8511b-5f96-4304-846d-b2b96690bef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:37.468322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8511b-5f96-4304-846d-b2b96690bef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:37.469134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8511b-5f96-4304-846d-b2b96690bef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:37.471487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8511b-5f96-4304-846d-b2b96690bef5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:37.509586 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 824/1630] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:37.511088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aabc47-564c-4d72-bd26-d4e88a3b8c39 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:37.527665 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aabc47-564c-4d72-bd26-d4e88a3b8c39 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:37.540982 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09569799-944b-4e98-8831-e867ceae2f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:37.544786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09569799-944b-4e98-8831-e867ceae2f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:37.555405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09569799-944b-4e98-8831-e867ceae2f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:37.555405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09569799-944b-4e98-8831-e867ceae2f3e admin admin] PATH_INFO: `/v3/projects/8a7621566d3d4db1ae6d359ef4ee3f0a/users/be8b2c792dbc4f3d9d04583e19529570/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:37.555405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09569799-944b-4e98-8831-e867ceae2f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7621566d3d4db1ae6d359ef4ee3f0a, user_id=be8b2c792dbc4f3d9d04583e19529570, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:37.556744 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09569799-944b-4e98-8831-e867ceae2f3e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:37.574534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 807/1631] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:37 2022] PUT /identity/v3/projects/8a7621566d3d4db1ae6d359ef4ee3f0a/users/be8b2c792dbc4f3d9d04583e19529570/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:46:37.584457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d1204-0998-4f12-b5b8-a4195bfedbae None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:37.584911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d1204-0998-4f12-b5b8-a4195bfedbae None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:37.585241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d1204-0998-4f12-b5b8-a4195bfedbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:37.619823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ce3d1204-0998-4f12-b5b8-a4195bfedbae None None] MFA Rules not processed for user `be8b2c792dbc4f3d9d04583e19529570`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:37.621290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ce3d1204-0998-4f12-b5b8-a4195bfedbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:37.654674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:38.125323 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 825/1632] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:37 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:38.135716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a9d60-eeb7-4d34-ae48-23fb159f57b1 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:38.135716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a9d60-eeb7-4d34-ae48-23fb159f57b1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:38.135716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a9d60-eeb7-4d34-ae48-23fb159f57b1 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:38.135716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 808/1633] 10.222.0.44 () {54 vars in 912 bytes} [Sun Sep 11 08:46:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:46:38.142533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0889f6b-4307-4484-a567-c40c17374b3f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:38.142533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0889f6b-4307-4484-a567-c40c17374b3f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:38.142533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0889f6b-4307-4484-a567-c40c17374b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:38.142533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a0889f6b-4307-4484-a567-c40c17374b3f None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:38.142533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a0889f6b-4307-4484-a567-c40c17374b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:38.143995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a9d60-eeb7-4d34-ae48-23fb159f57b1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:38.145717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 826/1634] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:46:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:38.145977 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a9d60-eeb7-4d34-ae48-23fb159f57b1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:38.145977 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66a926-16ef-4c1a-9948-ee2762cbfa55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:38.145977 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66a926-16ef-4c1a-9948-ee2762cbfa55 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:38.145977 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66a926-16ef-4c1a-9948-ee2762cbfa55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:38.146699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66a926-16ef-4c1a-9948-ee2762cbfa55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:38.146699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66a926-16ef-4c1a-9948-ee2762cbfa55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:38.146699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66a926-16ef-4c1a-9948-ee2762cbfa55 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:38.146699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 809/1635] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:37 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:46:38.147156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0889f6b-4307-4484-a567-c40c17374b3f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:38.147156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0889f6b-4307-4484-a567-c40c17374b3f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:38.147156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81691df9-7240-424f-bf3c-5e43b1d3b8eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:38.147156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81691df9-7240-424f-bf3c-5e43b1d3b8eb service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:38.147820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81691df9-7240-424f-bf3c-5e43b1d3b8eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:38.147820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81691df9-7240-424f-bf3c-5e43b1d3b8eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:38.158121 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81691df9-7240-424f-bf3c-5e43b1d3b8eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:38.160105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81691df9-7240-424f-bf3c-5e43b1d3b8eb service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:38.177017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 827/1636] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:38.513284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66a926-16ef-4c1a-9948-ee2762cbfa55 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:38.568576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81691df9-7240-424f-bf3c-5e43b1d3b8eb service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:38.671800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66a926-16ef-4c1a-9948-ee2762cbfa55 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:38.681681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8a65a-6abc-4089-a035-be032b2107e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:38.682448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8a65a-6abc-4089-a035-be032b2107e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:38.682720 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8a65a-6abc-4089-a035-be032b2107e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:38.682987 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8a65a-6abc-4089-a035-be032b2107e9 admin admin] PATH_INFO: `/v3/users/c0013ae7efed4f35a66a27e4ef17f290` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:38.694414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8a65a-6abc-4089-a035-be032b2107e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0013ae7efed4f35a66a27e4ef17f290)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:38.696683 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8a65a-6abc-4089-a035-be032b2107e9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:38.704779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81691df9-7240-424f-bf3c-5e43b1d3b8eb service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:38.711988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:38.711988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:38.711988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:38.714042 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] PATH_INFO: `/v3/users/d2d0b3a45c3049c58c1b7e5e91f8e2f4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:38.715575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2d0b3a45c3049c58c1b7e5e91f8e2f4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:38.719411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:46:38.719411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Sep 11 08:46:38.719411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: Sep 11 08:46:38.720897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:39.017260 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f5f8a65a-6abc-4089-a035-be032b2107e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0013ae7efed4f35a66a27e4ef17f290'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:39.019249 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:39.033060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f5f8a65a-6abc-4089-a035-be032b2107e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0013ae7efed4f35a66a27e4ef17f290'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:39.059589 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:39.078432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2d0b3a45c3049c58c1b7e5e91f8e2f4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:39.079878 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2d0b3a45c3049c58c1b7e5e91f8e2f4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:39.152641 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f5f8a65a-6abc-4089-a035-be032b2107e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0013ae7efed4f35a66a27e4ef17f290'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:39.153108 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f5f8a65a-6abc-4089-a035-be032b2107e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0013ae7efed4f35a66a27e4ef17f290'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:39.162971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f5f8a65a-6abc-4089-a035-be032b2107e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0013ae7efed4f35a66a27e4ef17f290'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:39.183817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 810/1637] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:46:38 2022] DELETE /identity/v3/users/c0013ae7efed4f35a66a27e4ef17f290 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:39.187304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b998ae-5828-48f7-8d4c-f94893b9b564 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:39.188053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b998ae-5828-48f7-8d4c-f94893b9b564 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:39.188919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b998ae-5828-48f7-8d4c-f94893b9b564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:39.214079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e5b998ae-5828-48f7-8d4c-f94893b9b564 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:39.216833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e5b998ae-5828-48f7-8d4c-f94893b9b564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:39.245654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2d0b3a45c3049c58c1b7e5e91f8e2f4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:39.266977 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2d0b3a45c3049c58c1b7e5e91f8e2f4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:39.296900 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2d0b3a45c3049c58c1b7e5e91f8e2f4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:39.311929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 811/1638] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:39.316964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 828/1639] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:46:38 2022] DELETE /identity/v3/users/d2d0b3a45c3049c58c1b7e5e91f8e2f4 => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:39.323573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815326db-0feb-40fa-a4f5-384780712721 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:39.324671 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815326db-0feb-40fa-a4f5-384780712721 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:39.324953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815326db-0feb-40fa-a4f5-384780712721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:39.332085 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:39.335295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:39.349587 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-815326db-0feb-40fa-a4f5-384780712721 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:39.350906 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-815326db-0feb-40fa-a4f5-384780712721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:39.419706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee94f2-4947-49a0-9d58-ecea1c614ecd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:39.424127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd9730-4b14-40cb-93b3-c98747214875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:39.424783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd9730-4b14-40cb-93b3-c98747214875 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:39.425055 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd9730-4b14-40cb-93b3-c98747214875 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:39.425333 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd9730-4b14-40cb-93b3-c98747214875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:39.474911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 812/1640] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:39.476467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815326db-0feb-40fa-a4f5-384780712721 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:39.478450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-815326db-0feb-40fa-a4f5-384780712721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:39.497065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815326db-0feb-40fa-a4f5-384780712721 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:39.503214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac64057-1668-471a-af31-af7cde64f264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:39.504136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac64057-1668-471a-af31-af7cde64f264 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:39.504517 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac64057-1668-471a-af31-af7cde64f264 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:39.504952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac64057-1668-471a-af31-af7cde64f264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:39.518630 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd9730-4b14-40cb-93b3-c98747214875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:39.519232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac64057-1668-471a-af31-af7cde64f264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:39.521608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd9730-4b14-40cb-93b3-c98747214875 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:39.523500 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac64057-1668-471a-af31-af7cde64f264 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:39.540767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 829/1641] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:39.542369 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 813/1642] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:40.607493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd9730-4b14-40cb-93b3-c98747214875 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:40.609236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-05bd9730-4b14-40cb-93b3-c98747214875 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:40.629861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd9730-4b14-40cb-93b3-c98747214875 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:40.639041 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:40.639912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:40.640248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:40.641457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] PATH_INFO: `/v3/projects/842fcece97a7440faab41cb09e1eb316` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:40.643807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=842fcece97a7440faab41cb09e1eb316)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:40.646163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:40.914861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '842fcece97a7440faab41cb09e1eb316', 'request_id': 'req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:40.919206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:40.966383 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac64057-1668-471a-af31-af7cde64f264 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:40.967885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bac64057-1668-471a-af31-af7cde64f264 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:40.981741 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac64057-1668-471a-af31-af7cde64f264 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:41.007355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '842fcece97a7440faab41cb09e1eb316', 'request_id': 'req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.016387 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '842fcece97a7440faab41cb09e1eb316', 'request_id': 'req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.036898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] The token cache is being invalidate because project 842fcece97a7440faab41cb09e1eb316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:46:41.037436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.038735 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.042976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 830/1643] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:46:40 2022] DELETE /identity/v3/projects/842fcece97a7440faab41cb09e1eb316 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:41.054978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:41.056506 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:41.062498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:41.087535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed4865-9dc2-48fa-b060-8877df590310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:41.088266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed4865-9dc2-48fa-b060-8877df590310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:41.088600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed4865-9dc2-48fa-b060-8877df590310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:41.088946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed4865-9dc2-48fa-b060-8877df590310 admin admin] PATH_INFO: `/v3/projects/0db8bf71fdf848d3acbbd165481f143b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:41.097916 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed4865-9dc2-48fa-b060-8877df590310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0db8bf71fdf848d3acbbd165481f143b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:41.105706 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-86ed4865-9dc2-48fa-b060-8877df590310 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 08:46:41.105706 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Sep 11 08:46:41.105706 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: Sep 11 08:46:41.111792 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed4865-9dc2-48fa-b060-8877df590310 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:41.151674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5da7e-c9f4-45df-bf4c-8cd3d19e4022 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:41.161476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17482883-b862-498f-9e5a-8ae9de8b7c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:41.162299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17482883-b862-498f-9e5a-8ae9de8b7c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:41.180026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17482883-b862-498f-9e5a-8ae9de8b7c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:41.180026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17482883-b862-498f-9e5a-8ae9de8b7c36 admin admin] PATH_INFO: `/v3/users/be8b2c792dbc4f3d9d04583e19529570` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:41.180026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17482883-b862-498f-9e5a-8ae9de8b7c36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be8b2c792dbc4f3d9d04583e19529570)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:41.180026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17482883-b862-498f-9e5a-8ae9de8b7c36 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:41.330585 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-86ed4865-9dc2-48fa-b060-8877df590310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0db8bf71fdf848d3acbbd165481f143b', 'request_id': 'req-86ed4865-9dc2-48fa-b060-8877df590310'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.428756 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-86ed4865-9dc2-48fa-b060-8877df590310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0db8bf71fdf848d3acbbd165481f143b', 'request_id': 'req-86ed4865-9dc2-48fa-b060-8877df590310'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.454554 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-86ed4865-9dc2-48fa-b060-8877df590310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0db8bf71fdf848d3acbbd165481f143b', 'request_id': 'req-86ed4865-9dc2-48fa-b060-8877df590310'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.492159 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17482883-b862-498f-9e5a-8ae9de8b7c36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be8b2c792dbc4f3d9d04583e19529570'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.494387 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17482883-b862-498f-9e5a-8ae9de8b7c36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be8b2c792dbc4f3d9d04583e19529570'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.520236 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-86ed4865-9dc2-48fa-b060-8877df590310 admin admin] The token cache is being invalidate because project 0db8bf71fdf848d3acbbd165481f143b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:46:41.520462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-86ed4865-9dc2-48fa-b060-8877df590310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.521213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-86ed4865-9dc2-48fa-b060-8877df590310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.527809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 814/1644] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:46:40 2022] DELETE /identity/v3/projects/0db8bf71fdf848d3acbbd165481f143b => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:46:41.532799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14897cc1-d0b2-490b-8b12-69d1c7a7108f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:41.532799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14897cc1-d0b2-490b-8b12-69d1c7a7108f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:41.533190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14897cc1-d0b2-490b-8b12-69d1c7a7108f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:41.595315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-14897cc1-d0b2-490b-8b12-69d1c7a7108f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:41.596689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-14897cc1-d0b2-490b-8b12-69d1c7a7108f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:41.620849 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17482883-b862-498f-9e5a-8ae9de8b7c36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be8b2c792dbc4f3d9d04583e19529570'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.636770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17482883-b862-498f-9e5a-8ae9de8b7c36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be8b2c792dbc4f3d9d04583e19529570'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.709592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17482883-b862-498f-9e5a-8ae9de8b7c36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be8b2c792dbc4f3d9d04583e19529570'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:41.732768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 831/1645] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:46:41 2022] DELETE /identity/v3/users/be8b2c792dbc4f3d9d04583e19529570 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:41.810565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 815/1646] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:46:41.819831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4471d7-72a1-4115-8a4a-51ead42d1a13 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:41.820325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4471d7-72a1-4115-8a4a-51ead42d1a13 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:41.824639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4471d7-72a1-4115-8a4a-51ead42d1a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:41.856448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4b4471d7-72a1-4115-8a4a-51ead42d1a13 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:41.859336 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4b4471d7-72a1-4115-8a4a-51ead42d1a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:41.873144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14897cc1-d0b2-490b-8b12-69d1c7a7108f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:41.875565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-14897cc1-d0b2-490b-8b12-69d1c7a7108f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:41.894209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14897cc1-d0b2-490b-8b12-69d1c7a7108f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:41.956440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 832/1647] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:41.958090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4471d7-72a1-4115-8a4a-51ead42d1a13 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:41.971204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4471d7-72a1-4115-8a4a-51ead42d1a13 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:41.985755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e9ddb-81d6-444a-8270-1b9bb42dd78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:41.986352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e9ddb-81d6-444a-8270-1b9bb42dd78f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:41.986638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e9ddb-81d6-444a-8270-1b9bb42dd78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:41.986914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e9ddb-81d6-444a-8270-1b9bb42dd78f admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:41.989384 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e9ddb-81d6-444a-8270-1b9bb42dd78f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:46:41.990704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e9ddb-81d6-444a-8270-1b9bb42dd78f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:41.993518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e9ddb-81d6-444a-8270-1b9bb42dd78f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:42.008946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5cddd-c717-429c-895e-a747c68dad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:42.010412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5cddd-c717-429c-895e-a747c68dad7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:42.010848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5cddd-c717-429c-895e-a747c68dad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:42.016365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5cddd-c717-429c-895e-a747c68dad7b admin admin] PATH_INFO: `/v3/users/ca42316192ec4c62b98ead2e7d659880` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:42.016365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5cddd-c717-429c-895e-a747c68dad7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca42316192ec4c62b98ead2e7d659880)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:42.017882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5cddd-c717-429c-895e-a747c68dad7b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:42.027389 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 833/1648] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:46:42.032944 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e9ddb-81d6-444a-8270-1b9bb42dd78f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:42.052965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e9ddb-81d6-444a-8270-1b9bb42dd78f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:42.067415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ad5a1-5969-47e0-bdaf-383cb4d520e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:42.068057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ad5a1-5969-47e0-bdaf-383cb4d520e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:42.068328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ad5a1-5969-47e0-bdaf-383cb4d520e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:42.068802 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ad5a1-5969-47e0-bdaf-383cb4d520e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:42.069470 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ad5a1-5969-47e0-bdaf-383cb4d520e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:42.071485 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ad5a1-5969-47e0-bdaf-383cb4d520e2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:42.204784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 834/1649] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:42 2022] POST /identity/v3/projects => generated 403 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:42.209313 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ad5a1-5969-47e0-bdaf-383cb4d520e2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:42.383928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ad5a1-5969-47e0-bdaf-383cb4d520e2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:42.385182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ecc5cddd-c717-429c-895e-a747c68dad7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca42316192ec4c62b98ead2e7d659880'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:42.396729 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660a1cb-f36c-4daf-a0f5-97e806159865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:42.401787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660a1cb-f36c-4daf-a0f5-97e806159865 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:42.402861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660a1cb-f36c-4daf-a0f5-97e806159865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:42.403142 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660a1cb-f36c-4daf-a0f5-97e806159865 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:42.403832 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660a1cb-f36c-4daf-a0f5-97e806159865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:42.407151 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660a1cb-f36c-4daf-a0f5-97e806159865 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:42.417753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ecc5cddd-c717-429c-895e-a747c68dad7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca42316192ec4c62b98ead2e7d659880'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:42.569025 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ecc5cddd-c717-429c-895e-a747c68dad7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca42316192ec4c62b98ead2e7d659880'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:42.569669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ecc5cddd-c717-429c-895e-a747c68dad7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca42316192ec4c62b98ead2e7d659880'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:42.603858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ecc5cddd-c717-429c-895e-a747c68dad7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca42316192ec4c62b98ead2e7d659880'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:42.610328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 835/1650] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:42 2022] POST /identity/v3/users => generated 302 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:46:42.612853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660a1cb-f36c-4daf-a0f5-97e806159865 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:42.615340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d660a1cb-f36c-4daf-a0f5-97e806159865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:42.637883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 816/1651] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:46:41 2022] DELETE /identity/v3/users/ca42316192ec4c62b98ead2e7d659880 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:42.658842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00785f22-1edc-4a04-b689-34ad54675bcd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:42.659363 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00785f22-1edc-4a04-b689-34ad54675bcd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:42.659815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00785f22-1edc-4a04-b689-34ad54675bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:42.663053 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660a1cb-f36c-4daf-a0f5-97e806159865 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:42.723580 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-00785f22-1edc-4a04-b689-34ad54675bcd None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:42.724857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-00785f22-1edc-4a04-b689-34ad54675bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:42.859383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d5a9b-31dd-432e-a8bb-c72b0856e599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:42.859383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d5a9b-31dd-432e-a8bb-c72b0856e599 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:42.859383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d5a9b-31dd-432e-a8bb-c72b0856e599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:42.859383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d5a9b-31dd-432e-a8bb-c72b0856e599 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:42.859771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d5a9b-31dd-432e-a8bb-c72b0856e599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:42.879639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d5a9b-31dd-432e-a8bb-c72b0856e599 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:42.907000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 836/1652] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:42.912411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d5a9b-31dd-432e-a8bb-c72b0856e599 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:42.925133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d5a9b-31dd-432e-a8bb-c72b0856e599 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:42.930904 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab262ca-6ee0-46f4-b6ea-c4d9d48b109a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:42.931801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab262ca-6ee0-46f4-b6ea-c4d9d48b109a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:42.932082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab262ca-6ee0-46f4-b6ea-c4d9d48b109a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:42.932350 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab262ca-6ee0-46f4-b6ea-c4d9d48b109a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:42.933083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab262ca-6ee0-46f4-b6ea-c4d9d48b109a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:42.935893 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab262ca-6ee0-46f4-b6ea-c4d9d48b109a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:42.962675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 837/1653] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:42.963661 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab262ca-6ee0-46f4-b6ea-c4d9d48b109a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:42.976007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 817/1654] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:42.983742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab262ca-6ee0-46f4-b6ea-c4d9d48b109a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:42.990034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c3ccd-3c2d-4717-8299-ec618da7f5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:42.990758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c3ccd-3c2d-4717-8299-ec618da7f5af admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:42.991208 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c3ccd-3c2d-4717-8299-ec618da7f5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:42.991553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c3ccd-3c2d-4717-8299-ec618da7f5af admin admin] PATH_INFO: `/v3/projects/c0b804d74cdf44328ab0446a2f485080/users/eaaddb757a134fd4803e0b73fb670fce/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:43.008925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00785f22-1edc-4a04-b689-34ad54675bcd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:43.010646 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-00785f22-1edc-4a04-b689-34ad54675bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:43.016519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:43.034403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c3ccd-3c2d-4717-8299-ec618da7f5af admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0b804d74cdf44328ab0446a2f485080, user_id=eaaddb757a134fd4803e0b73fb670fce, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:43.039281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c3ccd-3c2d-4717-8299-ec618da7f5af admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:43.114629 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00785f22-1edc-4a04-b689-34ad54675bcd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:43.126193 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 838/1655] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:42 2022] PUT /identity/v3/projects/c0b804d74cdf44328ab0446a2f485080/users/eaaddb757a134fd4803e0b73fb670fce/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:43.132012 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3199dc-5c4e-4dd1-8f11-e95a95cb962e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:43.132333 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3199dc-5c4e-4dd1-8f11-e95a95cb962e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:43.133856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3199dc-5c4e-4dd1-8f11-e95a95cb962e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:43.142921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:43.218655 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790f6ff-dfa1-43f6-a5fc-cab00e15c841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:43.219631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790f6ff-dfa1-43f6-a5fc-cab00e15c841 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:43.220094 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790f6ff-dfa1-43f6-a5fc-cab00e15c841 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:43.221810 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790f6ff-dfa1-43f6-a5fc-cab00e15c841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:43.221908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7f3199dc-5c4e-4dd1-8f11-e95a95cb962e None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:43.225035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7f3199dc-5c4e-4dd1-8f11-e95a95cb962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:43.347035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790f6ff-dfa1-43f6-a5fc-cab00e15c841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:43.350650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790f6ff-dfa1-43f6-a5fc-cab00e15c841 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:43.377109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 818/1656] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:46:43.386336 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 839/1657] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:43.430825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790f6ff-dfa1-43f6-a5fc-cab00e15c841 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:43.453828 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790f6ff-dfa1-43f6-a5fc-cab00e15c841 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:43.460349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbf93f-8fed-43f3-82a2-bfe7fc96d90a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:43.467454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbf93f-8fed-43f3-82a2-bfe7fc96d90a service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:43.467747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbf93f-8fed-43f3-82a2-bfe7fc96d90a service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:43.468369 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbf93f-8fed-43f3-82a2-bfe7fc96d90a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:43.490599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbf93f-8fed-43f3-82a2-bfe7fc96d90a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:43.495119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbf93f-8fed-43f3-82a2-bfe7fc96d90a service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:43.525875 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 819/1658] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:44.623783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3199dc-5c4e-4dd1-8f11-e95a95cb962e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:44.629986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7f3199dc-5c4e-4dd1-8f11-e95a95cb962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:44.657653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3199dc-5c4e-4dd1-8f11-e95a95cb962e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:44.666561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:44.671850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527e3e0-e263-4f90-82df-d2d93be14dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:44.673280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527e3e0-e263-4f90-82df-d2d93be14dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:44.677508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527e3e0-e263-4f90-82df-d2d93be14dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:44.678374 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527e3e0-e263-4f90-82df-d2d93be14dbc admin admin] PATH_INFO: `/v3/projects/8a7621566d3d4db1ae6d359ef4ee3f0a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:44.682703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527e3e0-e263-4f90-82df-d2d93be14dbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7621566d3d4db1ae6d359ef4ee3f0a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:44.692901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527e3e0-e263-4f90-82df-d2d93be14dbc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:44.944164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3527e3e0-e263-4f90-82df-d2d93be14dbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7621566d3d4db1ae6d359ef4ee3f0a', 'request_id': 'req-3527e3e0-e263-4f90-82df-d2d93be14dbc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:44.992493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:45.108633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3527e3e0-e263-4f90-82df-d2d93be14dbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7621566d3d4db1ae6d359ef4ee3f0a', 'request_id': 'req-3527e3e0-e263-4f90-82df-d2d93be14dbc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:45.129987 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3527e3e0-e263-4f90-82df-d2d93be14dbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7621566d3d4db1ae6d359ef4ee3f0a', 'request_id': 'req-3527e3e0-e263-4f90-82df-d2d93be14dbc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:45.205635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3527e3e0-e263-4f90-82df-d2d93be14dbc admin admin] The token cache is being invalidate because project 8a7621566d3d4db1ae6d359ef4ee3f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:46:45.206092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3527e3e0-e263-4f90-82df-d2d93be14dbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:45.211202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3527e3e0-e263-4f90-82df-d2d93be14dbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:45.232815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 840/1659] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:46:44 2022] DELETE /identity/v3/projects/8a7621566d3d4db1ae6d359ef4ee3f0a => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:45.237964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894da84-4e8f-4369-b140-0b1018841611 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:45.238507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894da84-4e8f-4369-b140-0b1018841611 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:45.239124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894da84-4e8f-4369-b140-0b1018841611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:45.287614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a894da84-4e8f-4369-b140-0b1018841611 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:45.293765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a894da84-4e8f-4369-b140-0b1018841611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:45.508615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 820/1660] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:46:45.519326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d49721-fa7b-4826-8752-345f98e4ff4a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:45.519918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d49721-fa7b-4826-8752-345f98e4ff4a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:45.521487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d49721-fa7b-4826-8752-345f98e4ff4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:45.565748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-39d49721-fa7b-4826-8752-345f98e4ff4a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:45.567295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-39d49721-fa7b-4826-8752-345f98e4ff4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:45.586639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 841/1661] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:45.591110 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894da84-4e8f-4369-b140-0b1018841611 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:45.604730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894da84-4e8f-4369-b140-0b1018841611 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:45.614893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32e36e-2b32-4944-97e9-43803d4b79d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:45.617075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32e36e-2b32-4944-97e9-43803d4b79d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:45.617385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32e36e-2b32-4944-97e9-43803d4b79d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:45.617655 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32e36e-2b32-4944-97e9-43803d4b79d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:45.619335 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32e36e-2b32-4944-97e9-43803d4b79d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:46:45.619660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32e36e-2b32-4944-97e9-43803d4b79d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:45.621903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32e36e-2b32-4944-97e9-43803d4b79d4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:45.651889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 821/1662] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:46:45.656933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d49721-fa7b-4826-8752-345f98e4ff4a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:45.667622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d49721-fa7b-4826-8752-345f98e4ff4a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:45.675834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8841f3-8942-45f6-a31d-3c01b14b71a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:45.676582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8841f3-8942-45f6-a31d-3c01b14b71a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:45.676938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8841f3-8942-45f6-a31d-3c01b14b71a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:45.677286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8841f3-8942-45f6-a31d-3c01b14b71a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:45.678208 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8841f3-8942-45f6-a31d-3c01b14b71a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:45.682872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8841f3-8942-45f6-a31d-3c01b14b71a1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:45.696145 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32e36e-2b32-4944-97e9-43803d4b79d4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:45.698412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3d32e36e-2b32-4944-97e9-43803d4b79d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:45.727841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32e36e-2b32-4944-97e9-43803d4b79d4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:45.754669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724133a2-f768-4520-a22b-4db53e7f78d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:45.755625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724133a2-f768-4520-a22b-4db53e7f78d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:45.756101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724133a2-f768-4520-a22b-4db53e7f78d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:45.756498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724133a2-f768-4520-a22b-4db53e7f78d5 admin admin] PATH_INFO: `/v3/projects/f96c7a52c30e4043908a94348eb6af9b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:45.757978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724133a2-f768-4520-a22b-4db53e7f78d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96c7a52c30e4043908a94348eb6af9b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:45.763538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724133a2-f768-4520-a22b-4db53e7f78d5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:45.876361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 842/1663] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:45 2022] POST /identity/v3/projects => generated 365 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:45.878398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8841f3-8942-45f6-a31d-3c01b14b71a1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:46.173375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-724133a2-f768-4520-a22b-4db53e7f78d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96c7a52c30e4043908a94348eb6af9b', 'request_id': 'req-724133a2-f768-4520-a22b-4db53e7f78d5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:46.208150 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8841f3-8942-45f6-a31d-3c01b14b71a1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:46.270612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:46.335607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-724133a2-f768-4520-a22b-4db53e7f78d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96c7a52c30e4043908a94348eb6af9b', 'request_id': 'req-724133a2-f768-4520-a22b-4db53e7f78d5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:46.362465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-724133a2-f768-4520-a22b-4db53e7f78d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96c7a52c30e4043908a94348eb6af9b', 'request_id': 'req-724133a2-f768-4520-a22b-4db53e7f78d5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:46.392920 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76789c3-d65e-4024-a4c2-85e548ea0639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:46.393844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76789c3-d65e-4024-a4c2-85e548ea0639 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:46.394351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76789c3-d65e-4024-a4c2-85e548ea0639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:46.394760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76789c3-d65e-4024-a4c2-85e548ea0639 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:46.395644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76789c3-d65e-4024-a4c2-85e548ea0639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:46.399098 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76789c3-d65e-4024-a4c2-85e548ea0639 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:46.484803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-724133a2-f768-4520-a22b-4db53e7f78d5 admin admin] The token cache is being invalidate because project f96c7a52c30e4043908a94348eb6af9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:46:46.488037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-724133a2-f768-4520-a22b-4db53e7f78d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:46.492462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-724133a2-f768-4520-a22b-4db53e7f78d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:46.501798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 822/1664] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:46:45 2022] DELETE /identity/v3/projects/f96c7a52c30e4043908a94348eb6af9b => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:46:46.513548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7e862-1540-47b0-8ee9-8f80e94818dd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:46.516478 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7e862-1540-47b0-8ee9-8f80e94818dd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:46.518056 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7e862-1540-47b0-8ee9-8f80e94818dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:46.537343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 843/1665] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:45 2022] POST /identity/v3/users => generated 283 bytes in 660 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:46.546762 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76789c3-d65e-4024-a4c2-85e548ea0639 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:46.548775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a76789c3-d65e-4024-a4c2-85e548ea0639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:46.572514 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76789c3-d65e-4024-a4c2-85e548ea0639 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:46.602086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7433e-53f7-43a5-a30f-5906b0df61c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:46.602928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7433e-53f7-43a5-a30f-5906b0df61c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:46.611988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7433e-53f7-43a5-a30f-5906b0df61c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:46.611988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7433e-53f7-43a5-a30f-5906b0df61c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:46.611988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7433e-53f7-43a5-a30f-5906b0df61c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:46.611988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7433e-53f7-43a5-a30f-5906b0df61c8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:46.646687 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-21f7e862-1540-47b0-8ee9-8f80e94818dd None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:46.649414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-21f7e862-1540-47b0-8ee9-8f80e94818dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:46.700698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 823/1666] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:46.709081 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb59ee-94fc-4797-8e8c-e09c0da088dd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:46.710667 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 844/1667] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:46.711217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb59ee-94fc-4797-8e8c-e09c0da088dd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:46.711926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb59ee-94fc-4797-8e8c-e09c0da088dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:46.713026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7433e-53f7-43a5-a30f-5906b0df61c8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:46.770548 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7433e-53f7-43a5-a30f-5906b0df61c8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:46.788837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5cbb59ee-94fc-4797-8e8c-e09c0da088dd None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:46.792220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5cbb59ee-94fc-4797-8e8c-e09c0da088dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:46.798900 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec369a8-3059-4bea-be01-bf6a8204cb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:46.799540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec369a8-3059-4bea-be01-bf6a8204cb55 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:46.799820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec369a8-3059-4bea-be01-bf6a8204cb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:46.800108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec369a8-3059-4bea-be01-bf6a8204cb55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:46.806286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec369a8-3059-4bea-be01-bf6a8204cb55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:46.806286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec369a8-3059-4bea-be01-bf6a8204cb55 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:46.809621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 824/1668] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:46:46.812331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb59ee-94fc-4797-8e8c-e09c0da088dd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:46.826391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb59ee-94fc-4797-8e8c-e09c0da088dd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:46.836854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 845/1669] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:46.839921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec369a8-3059-4bea-be01-bf6a8204cb55 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:46.842534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea82617-b565-42ee-bff4-6f773157b150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:46.843203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea82617-b565-42ee-bff4-6f773157b150 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:46.843464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea82617-b565-42ee-bff4-6f773157b150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:46.843735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea82617-b565-42ee-bff4-6f773157b150 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:46.844392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea82617-b565-42ee-bff4-6f773157b150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:46:46.844700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea82617-b565-42ee-bff4-6f773157b150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:46.847048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea82617-b565-42ee-bff4-6f773157b150 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:46.869011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec369a8-3059-4bea-be01-bf6a8204cb55 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:46.893402 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 825/1670] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:46:46.896210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea82617-b565-42ee-bff4-6f773157b150 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:46.896928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24aa58-328e-4e3c-9477-f0c8549a8255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:46.898960 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24aa58-328e-4e3c-9477-f0c8549a8255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:46.900511 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24aa58-328e-4e3c-9477-f0c8549a8255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:46.902574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24aa58-328e-4e3c-9477-f0c8549a8255 admin admin] PATH_INFO: `/v3/projects/b13af905788d42c39b3233055a34589d/users/549f76f68dfa441095c74bbb6ab02d32/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:46.932792 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea82617-b565-42ee-bff4-6f773157b150 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:46.940957 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c1909-064d-4bd5-9623-17d6db985217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:46.941966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c1909-064d-4bd5-9623-17d6db985217 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:46.942563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c1909-064d-4bd5-9623-17d6db985217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:46.944005 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c1909-064d-4bd5-9623-17d6db985217 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:46.945474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c1909-064d-4bd5-9623-17d6db985217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:46.947799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24aa58-328e-4e3c-9477-f0c8549a8255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b13af905788d42c39b3233055a34589d, user_id=549f76f68dfa441095c74bbb6ab02d32, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:46.948037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c1909-064d-4bd5-9623-17d6db985217 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:46.958951 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24aa58-328e-4e3c-9477-f0c8549a8255 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:47.046107 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:47.084270 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 846/1671] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:46 2022] PUT /identity/v3/projects/b13af905788d42c39b3233055a34589d/users/549f76f68dfa441095c74bbb6ab02d32/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:47.111863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077dfde1-b540-441c-ac72-cdbb69db8ba2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:47.112440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077dfde1-b540-441c-ac72-cdbb69db8ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:47.112541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 826/1672] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:46 2022] POST /identity/v3/projects => generated 373 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:47.112974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077dfde1-b540-441c-ac72-cdbb69db8ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:47.137127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c1909-064d-4bd5-9623-17d6db985217 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:47.256091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-077dfde1-b540-441c-ac72-cdbb69db8ba2 None None] MFA Rules not processed for user `549f76f68dfa441095c74bbb6ab02d32`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:47.258700 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-077dfde1-b540-441c-ac72-cdbb69db8ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:47.396228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c1909-064d-4bd5-9623-17d6db985217 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:47.409973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d37e5-122c-4095-920c-7e0998b518ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:47.411701 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d37e5-122c-4095-920c-7e0998b518ef admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:47.412507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d37e5-122c-4095-920c-7e0998b518ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:47.413150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d37e5-122c-4095-920c-7e0998b518ef admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:47.414805 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d37e5-122c-4095-920c-7e0998b518ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:47.419883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d37e5-122c-4095-920c-7e0998b518ef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:47.578369 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 847/1673] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 11 08:46:47.580028 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077dfde1-b540-441c-ac72-cdbb69db8ba2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:47.599917 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 827/1674] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:47 2022] POST /identity/v3/users => generated 287 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:47.604619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d37e5-122c-4095-920c-7e0998b518ef admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:47.623362 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d37e5-122c-4095-920c-7e0998b518ef admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:47.633810 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edecd1b-5314-4cc7-b592-53379ec950a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:47.634995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edecd1b-5314-4cc7-b592-53379ec950a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:47.635321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077dfde1-b540-441c-ac72-cdbb69db8ba2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:47.635628 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edecd1b-5314-4cc7-b592-53379ec950a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:47.636717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edecd1b-5314-4cc7-b592-53379ec950a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:47.638936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edecd1b-5314-4cc7-b592-53379ec950a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:47.641205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edecd1b-5314-4cc7-b592-53379ec950a4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:47.655241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 828/1675] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:46:47.659964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c60386-4dfa-42ef-bef1-0bf19f2e9584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:47.661019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c60386-4dfa-42ef-bef1-0bf19f2e9584 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:47.661486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c60386-4dfa-42ef-bef1-0bf19f2e9584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:47.661954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c60386-4dfa-42ef-bef1-0bf19f2e9584 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:47.663148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c60386-4dfa-42ef-bef1-0bf19f2e9584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:47.672314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edecd1b-5314-4cc7-b592-53379ec950a4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:47.675683 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c60386-4dfa-42ef-bef1-0bf19f2e9584 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:47.686862 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edecd1b-5314-4cc7-b592-53379ec950a4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:47.699543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ba5e4-6dd6-40da-8540-c0a625c06aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:47.700595 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ba5e4-6dd6-40da-8540-c0a625c06aef admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:47.701093 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ba5e4-6dd6-40da-8540-c0a625c06aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:47.701608 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ba5e4-6dd6-40da-8540-c0a625c06aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:47.702553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ba5e4-6dd6-40da-8540-c0a625c06aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:47.704556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ba5e4-6dd6-40da-8540-c0a625c06aef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:47.722995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 829/1676] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:47.730720 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ba5e4-6dd6-40da-8540-c0a625c06aef admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:47.748040 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ba5e4-6dd6-40da-8540-c0a625c06aef admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:47.754336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c85624-61c0-4a1d-a0bb-0622dd239b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:47.755278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c85624-61c0-4a1d-a0bb-0622dd239b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:47.756107 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c85624-61c0-4a1d-a0bb-0622dd239b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:47.756632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c85624-61c0-4a1d-a0bb-0622dd239b66 admin admin] PATH_INFO: `/v3/projects/fff5782a0d7e43c3a2d610850528fd51/users/a8d4ad5ae33a42b2bad82844cee4d02e/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:47.783780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c85624-61c0-4a1d-a0bb-0622dd239b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff5782a0d7e43c3a2d610850528fd51, user_id=a8d4ad5ae33a42b2bad82844cee4d02e, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:47.792541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c85624-61c0-4a1d-a0bb-0622dd239b66 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:47.874678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 848/1677] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:47 2022] POST /identity/v3/projects => generated 365 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:47.879842 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c60386-4dfa-42ef-bef1-0bf19f2e9584 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:47.896487 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 830/1678] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:47 2022] PUT /identity/v3/projects/fff5782a0d7e43c3a2d610850528fd51/users/a8d4ad5ae33a42b2bad82844cee4d02e/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:47.913485 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624089d-380e-4b86-809e-7b5838d1a17e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:47.913700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624089d-380e-4b86-809e-7b5838d1a17e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:47.914061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624089d-380e-4b86-809e-7b5838d1a17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:47.988534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7624089d-380e-4b86-809e-7b5838d1a17e None None] MFA Rules not processed for user `a8d4ad5ae33a42b2bad82844cee4d02e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:47.989881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7624089d-380e-4b86-809e-7b5838d1a17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:47.994842 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:48.063317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c60386-4dfa-42ef-bef1-0bf19f2e9584 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:48.223003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 831/1679] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:46:48.228096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624089d-380e-4b86-809e-7b5838d1a17e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:48.276220 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce15802-6517-42df-ac4e-046bc3686c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:48.277394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce15802-6517-42df-ac4e-046bc3686c41 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:48.277668 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce15802-6517-42df-ac4e-046bc3686c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:48.283042 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce15802-6517-42df-ac4e-046bc3686c41 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:48.285289 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce15802-6517-42df-ac4e-046bc3686c41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:48.287704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce15802-6517-42df-ac4e-046bc3686c41 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:48.377930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624089d-380e-4b86-809e-7b5838d1a17e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:48.399795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaeab1f-e6ed-432e-8d1d-05eef786dad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:48.401222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeab1f-e6ed-432e-8d1d-05eef786dad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:48.401565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeab1f-e6ed-432e-8d1d-05eef786dad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:48.402608 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeab1f-e6ed-432e-8d1d-05eef786dad5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:48.403491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaeab1f-e6ed-432e-8d1d-05eef786dad5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:48.405627 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaeab1f-e6ed-432e-8d1d-05eef786dad5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:48.454353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 849/1680] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:47 2022] POST /identity/v3/users => generated 289 bytes in 575 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:48.458740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce15802-6517-42df-ac4e-046bc3686c41 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:48.490206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce15802-6517-42df-ac4e-046bc3686c41 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:48.504840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6c813-ab4e-4924-bccb-b08fcbd4d3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:48.508575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6c813-ab4e-4924-bccb-b08fcbd4d3cb admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:48.509166 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6c813-ab4e-4924-bccb-b08fcbd4d3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:48.509483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6c813-ab4e-4924-bccb-b08fcbd4d3cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:48.511823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6c813-ab4e-4924-bccb-b08fcbd4d3cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:48.518777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6c813-ab4e-4924-bccb-b08fcbd4d3cb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:48.559892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 850/1681] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:48.566476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6c813-ab4e-4924-bccb-b08fcbd4d3cb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:48.581567 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6c813-ab4e-4924-bccb-b08fcbd4d3cb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:48.589197 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a535a-b371-44d5-a333-732a1fee4671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:48.591166 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a535a-b371-44d5-a333-732a1fee4671 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:48.595775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a535a-b371-44d5-a333-732a1fee4671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:48.595775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a535a-b371-44d5-a333-732a1fee4671 admin admin] PATH_INFO: `/v3/projects/efaaa4f11d664a09a81ddd728c564f73/users/4eaade9548fb48a085f29dc57ffbe49e/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:48.604649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 832/1682] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:48 2022] POST /identity/v3/projects => generated 371 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:48.611561 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaeab1f-e6ed-432e-8d1d-05eef786dad5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:48.637903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a535a-b371-44d5-a333-732a1fee4671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efaaa4f11d664a09a81ddd728c564f73, user_id=4eaade9548fb48a085f29dc57ffbe49e, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:48.641541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a535a-b371-44d5-a333-732a1fee4671 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:48.788494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaeab1f-e6ed-432e-8d1d-05eef786dad5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:48.806078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791a3f1-9941-4b27-af15-8fefa7d4f1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:48.806866 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791a3f1-9941-4b27-af15-8fefa7d4f1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:48.807354 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791a3f1-9941-4b27-af15-8fefa7d4f1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:48.807855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791a3f1-9941-4b27-af15-8fefa7d4f1f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:48.809142 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791a3f1-9941-4b27-af15-8fefa7d4f1f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:48.809869 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 851/1683] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:48 2022] PUT /identity/v3/projects/efaaa4f11d664a09a81ddd728c564f73/users/4eaade9548fb48a085f29dc57ffbe49e/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:48.813650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791a3f1-9941-4b27-af15-8fefa7d4f1f2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:48.828686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8975d76-df79-430e-b5db-40bf232534ea None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:48.829492 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8975d76-df79-430e-b5db-40bf232534ea None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:48.830831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8975d76-df79-430e-b5db-40bf232534ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:48.929606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b8975d76-df79-430e-b5db-40bf232534ea None None] MFA Rules not processed for user `00407384465c41848ad28a15729a95d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:48.933116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b8975d76-df79-430e-b5db-40bf232534ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:48.957332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 833/1684] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:48 2022] POST /identity/v3/users => generated 292 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:48.963874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791a3f1-9941-4b27-af15-8fefa7d4f1f2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:49.020214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791a3f1-9941-4b27-af15-8fefa7d4f1f2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:49.040704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:49.178189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 852/1685] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:49.178189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd1274-def7-4afc-be69-9ef9361c0ac1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:49.178189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd1274-def7-4afc-be69-9ef9361c0ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:49.178189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd1274-def7-4afc-be69-9ef9361c0ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:49.234616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84658ed6-445d-411c-a597-09ace8a8c1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:49.235155 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84658ed6-445d-411c-a597-09ace8a8c1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:49.235449 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84658ed6-445d-411c-a597-09ace8a8c1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:49.235750 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84658ed6-445d-411c-a597-09ace8a8c1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:49.237553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84658ed6-445d-411c-a597-09ace8a8c1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:49.239330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84658ed6-445d-411c-a597-09ace8a8c1b7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:49.252452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-abdd1274-def7-4afc-be69-9ef9361c0ac1 None None] MFA Rules not processed for user `eaaddb757a134fd4803e0b73fb670fce`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:49.255746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-abdd1274-def7-4afc-be69-9ef9361c0ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:49.267249 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 834/1686] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:49.268164 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84658ed6-445d-411c-a597-09ace8a8c1b7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:49.288590 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84658ed6-445d-411c-a597-09ace8a8c1b7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:49.307050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed75c53-2451-49e3-99f7-69e697aaadc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:49.307691 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed75c53-2451-49e3-99f7-69e697aaadc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:49.308076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed75c53-2451-49e3-99f7-69e697aaadc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:49.308367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed75c53-2451-49e3-99f7-69e697aaadc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:49.309045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed75c53-2451-49e3-99f7-69e697aaadc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:49.310833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed75c53-2451-49e3-99f7-69e697aaadc7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:49.331063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 835/1687] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:49.335996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591edfac-703b-4c21-a7df-3edeb42dee88 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:49.335996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591edfac-703b-4c21-a7df-3edeb42dee88 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:49.335996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591edfac-703b-4c21-a7df-3edeb42dee88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:49.344293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:49.388318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-591edfac-703b-4c21-a7df-3edeb42dee88 None None] MFA Rules not processed for user `00407384465c41848ad28a15729a95d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:49.388318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-591edfac-703b-4c21-a7df-3edeb42dee88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:49.404498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 836/1688] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:49.406117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591edfac-703b-4c21-a7df-3edeb42dee88 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:49.430358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591edfac-703b-4c21-a7df-3edeb42dee88 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:49.446876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273233f2-27b8-4964-87fd-af90f97eca5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:49.448033 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273233f2-27b8-4964-87fd-af90f97eca5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:49.448643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273233f2-27b8-4964-87fd-af90f97eca5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:49.449130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273233f2-27b8-4964-87fd-af90f97eca5d admin admin] PATH_INFO: `/v3/projects/efaaa4f11d664a09a81ddd728c564f73/users/4eaade9548fb48a085f29dc57ffbe49e/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:49.451505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273233f2-27b8-4964-87fd-af90f97eca5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=efaaa4f11d664a09a81ddd728c564f73, user_id=4eaade9548fb48a085f29dc57ffbe49e, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:49.453139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 853/1689] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:49 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:49.454875 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273233f2-27b8-4964-87fd-af90f97eca5d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:49.458611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd1274-def7-4afc-be69-9ef9361c0ac1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:49.510684 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 837/1690] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:49 2022] PUT /identity/v3/projects/efaaa4f11d664a09a81ddd728c564f73/users/4eaade9548fb48a085f29dc57ffbe49e/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:49.534392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273233f2-27b8-4964-87fd-af90f97eca5d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:49.535878 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd1274-def7-4afc-be69-9ef9361c0ac1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:49.537214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-273233f2-27b8-4964-87fd-af90f97eca5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:49.674376 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5e78b-aa93-4dee-9c75-bb6cdecbf198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:49.685913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5e78b-aa93-4dee-9c75-bb6cdecbf198 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:49.686378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5e78b-aa93-4dee-9c75-bb6cdecbf198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:49.686651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5e78b-aa93-4dee-9c75-bb6cdecbf198 admin admin] PATH_INFO: `/v3/projects/6a139dc8eff74531863f0b9cd1248915/users/bf19911159be4e46aa3b2a290f2cb638/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:49.727593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5e78b-aa93-4dee-9c75-bb6cdecbf198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a139dc8eff74531863f0b9cd1248915, user_id=bf19911159be4e46aa3b2a290f2cb638, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:49.743297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5e78b-aa93-4dee-9c75-bb6cdecbf198 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:49.774232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273233f2-27b8-4964-87fd-af90f97eca5d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:49.789631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70944f07-96d0-4753-81da-b090ef787dac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:49.791425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70944f07-96d0-4753-81da-b090ef787dac service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:49.792197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70944f07-96d0-4753-81da-b090ef787dac service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:49.793675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70944f07-96d0-4753-81da-b090ef787dac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:49.863361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 854/1691] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:49 2022] PUT /identity/v3/projects/6a139dc8eff74531863f0b9cd1248915/users/bf19911159be4e46aa3b2a290f2cb638/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:49.863361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5e78b-aa93-4dee-9c75-bb6cdecbf198 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:49.869726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-57d5e78b-aa93-4dee-9c75-bb6cdecbf198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:50.096401 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70944f07-96d0-4753-81da-b090ef787dac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:50.099780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70944f07-96d0-4753-81da-b090ef787dac service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:50.114275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:50.150951 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 838/1692] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:50.155564 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5e78b-aa93-4dee-9c75-bb6cdecbf198 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:50.158694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9f5af-e252-4bdb-9103-7b59a7ed8fc8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:50.159202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9f5af-e252-4bdb-9103-7b59a7ed8fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:50.160479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9f5af-e252-4bdb-9103-7b59a7ed8fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:50.174005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d3505-5105-4972-b216-fa729ba03f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:50.174683 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d3505-5105-4972-b216-fa729ba03f6a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:50.174965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d3505-5105-4972-b216-fa729ba03f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:50.175234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d3505-5105-4972-b216-fa729ba03f6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:50.175895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d3505-5105-4972-b216-fa729ba03f6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:50.177962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d3505-5105-4972-b216-fa729ba03f6a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:50.262181 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fde9f5af-e252-4bdb-9103-7b59a7ed8fc8 None None] MFA Rules not processed for user `4eaade9548fb48a085f29dc57ffbe49e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:50.264847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fde9f5af-e252-4bdb-9103-7b59a7ed8fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:50.376836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 855/1693] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:46:49 2022] POST /identity/v3/projects => generated 405 bytes in 514 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:50.382780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d3505-5105-4972-b216-fa729ba03f6a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:50.507184 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:50.544295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d3505-5105-4972-b216-fa729ba03f6a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:50.560807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6a733-655c-439f-805d-36de4b91985c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:50.562331 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6a733-655c-439f-805d-36de4b91985c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:50.564318 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6a733-655c-439f-805d-36de4b91985c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:50.565112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6a733-655c-439f-805d-36de4b91985c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:50.566550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6a733-655c-439f-805d-36de4b91985c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:50.570562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6a733-655c-439f-805d-36de4b91985c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:50.580568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 839/1694] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:50.585825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9f5af-e252-4bdb-9103-7b59a7ed8fc8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:50.612504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 856/1695] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:50.612504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6a733-655c-439f-805d-36de4b91985c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:50.619655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-eaa6a733-655c-439f-805d-36de4b91985c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:50.640985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9f5af-e252-4bdb-9103-7b59a7ed8fc8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:50.666211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddab9c-40cf-42de-a767-6d59cfb71f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:50.676843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddab9c-40cf-42de-a767-6d59cfb71f59 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:50.678763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddab9c-40cf-42de-a767-6d59cfb71f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:50.678763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddab9c-40cf-42de-a767-6d59cfb71f59 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:50.681660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ddab9c-40cf-42de-a767-6d59cfb71f59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:50.692524 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ddab9c-40cf-42de-a767-6d59cfb71f59 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:50.842643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6a733-655c-439f-805d-36de4b91985c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:50.865398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c8e8d-3d84-4bdb-8666-eab010fcfb63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:50.867077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c8e8d-3d84-4bdb-8666-eab010fcfb63 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:50.868218 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c8e8d-3d84-4bdb-8666-eab010fcfb63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:50.868218 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c8e8d-3d84-4bdb-8666-eab010fcfb63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:50.894835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 840/1696] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:46:50 2022] POST /identity/v3/users => generated 309 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:46:50.900012 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddab9c-40cf-42de-a767-6d59cfb71f59 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:50.942938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddab9c-40cf-42de-a767-6d59cfb71f59 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:50.960299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cacdfe-a73a-47f0-bfb7-000be21f304c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:50.961511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cacdfe-a73a-47f0-bfb7-000be21f304c admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:50.962661 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cacdfe-a73a-47f0-bfb7-000be21f304c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:50.962788 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cacdfe-a73a-47f0-bfb7-000be21f304c admin admin] PATH_INFO: `/v3/projects/6a139dc8eff74531863f0b9cd1248915/users/bf19911159be4e46aa3b2a290f2cb638/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:50.967852 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cacdfe-a73a-47f0-bfb7-000be21f304c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a139dc8eff74531863f0b9cd1248915, user_id=bf19911159be4e46aa3b2a290f2cb638, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:50.976100 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cacdfe-a73a-47f0-bfb7-000be21f304c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:51.008357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 841/1697] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:50 2022] PUT /identity/v3/projects/6a139dc8eff74531863f0b9cd1248915/users/bf19911159be4e46aa3b2a290f2cb638/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:51.011002 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cacdfe-a73a-47f0-bfb7-000be21f304c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:51.035789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c8e8d-3d84-4bdb-8666-eab010fcfb63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:51.039240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c8e8d-3d84-4bdb-8666-eab010fcfb63 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:51.069390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 857/1698] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:51.073388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1b89a-cab4-473e-a808-cc73bef72a5e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:51.074565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1b89a-cab4-473e-a808-cc73bef72a5e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:51.075041 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1b89a-cab4-473e-a808-cc73bef72a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:51.147851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-45c1b89a-cab4-473e-a808-cc73bef72a5e None None] MFA Rules not processed for user `bf19911159be4e46aa3b2a290f2cb638`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:51.149630 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-45c1b89a-cab4-473e-a808-cc73bef72a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:51.160095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cacdfe-a73a-47f0-bfb7-000be21f304c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:51.166579 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b520ed-d414-4839-b089-ce62f707f617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:51.168204 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b520ed-d414-4839-b089-ce62f707f617 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:51.168840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b520ed-d414-4839-b089-ce62f707f617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:51.169279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b520ed-d414-4839-b089-ce62f707f617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:51.170257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b520ed-d414-4839-b089-ce62f707f617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:51.172215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b520ed-d414-4839-b089-ce62f707f617 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:51.189380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 842/1699] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:51.204688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b520ed-d414-4839-b089-ce62f707f617 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:51.245417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b520ed-d414-4839-b089-ce62f707f617 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:51.261838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3d0ac-bdc8-4708-8438-e6cd994465b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:51.261838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3d0ac-bdc8-4708-8438-e6cd994465b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:51.262128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3d0ac-bdc8-4708-8438-e6cd994465b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:51.263673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3d0ac-bdc8-4708-8438-e6cd994465b5 admin admin] PATH_INFO: `/v3/projects/7d4b0502715d43baa39c77f6d0fa46b3/users/571506ba11a7489797ae2c2013c50e68/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:51.288216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:51.312942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 858/1700] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:51.319103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3d0ac-bdc8-4708-8438-e6cd994465b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4b0502715d43baa39c77f6d0fa46b3, user_id=571506ba11a7489797ae2c2013c50e68, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:51.326934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3d0ac-bdc8-4708-8438-e6cd994465b5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:51.376908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1b89a-cab4-473e-a808-cc73bef72a5e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:51.378523 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-45c1b89a-cab4-473e-a808-cc73bef72a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:51.460096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 843/1701] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:51 2022] PUT /identity/v3/projects/7d4b0502715d43baa39c77f6d0fa46b3/users/571506ba11a7489797ae2c2013c50e68/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:51.470461 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:51.477740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca52ca-7f9e-44b3-a9c3-0c5d14f56318 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:51.480920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca52ca-7f9e-44b3-a9c3-0c5d14f56318 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:51.483808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca52ca-7f9e-44b3-a9c3-0c5d14f56318 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:51.503496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 844/1702] 10.222.0.44 () {54 vars in 912 bytes} [Sun Sep 11 08:46:51 2022] GET /identity => generated 270 bytes in 27 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:46:51.503496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca52ca-7f9e-44b3-a9c3-0c5d14f56318 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:51.521188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1b89a-cab4-473e-a808-cc73bef72a5e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:51.624325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b595f3-17dd-445d-8f8b-20d57fa49a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:51.624880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b595f3-17dd-445d-8f8b-20d57fa49a7b service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:51.625155 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b595f3-17dd-445d-8f8b-20d57fa49a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:51.625413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b595f3-17dd-445d-8f8b-20d57fa49a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:51.737879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca52ca-7f9e-44b3-a9c3-0c5d14f56318 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:51.751828 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d12f5-8aab-462f-89c2-ff31b606f5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:51.755455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d12f5-8aab-462f-89c2-ff31b606f5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:51.756866 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d12f5-8aab-462f-89c2-ff31b606f5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:51.757140 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b595f3-17dd-445d-8f8b-20d57fa49a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:51.758580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b595f3-17dd-445d-8f8b-20d57fa49a7b service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:51.774137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d12f5-8aab-462f-89c2-ff31b606f5b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:51.779611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d12f5-8aab-462f-89c2-ff31b606f5b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:51.783205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d12f5-8aab-462f-89c2-ff31b606f5b0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:51.784209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 859/1703] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:46:51.785859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bf7e6-70ee-4b0d-a0b5-d4c34c3dde6f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:51.786223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bf7e6-70ee-4b0d-a0b5-d4c34c3dde6f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:51.786507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bf7e6-70ee-4b0d-a0b5-d4c34c3dde6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:51.826124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 845/1704] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:46:51.829381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d12f5-8aab-462f-89c2-ff31b606f5b0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:51.855151 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d12f5-8aab-462f-89c2-ff31b606f5b0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:51.870811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14adab9-02eb-4029-8f5a-ac116f8d8c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:51.877959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14adab9-02eb-4029-8f5a-ac116f8d8c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:51.877959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14adab9-02eb-4029-8f5a-ac116f8d8c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:51.877959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14adab9-02eb-4029-8f5a-ac116f8d8c37 admin admin] PATH_INFO: `/v3/projects/7d4b0502715d43baa39c77f6d0fa46b3/users/571506ba11a7489797ae2c2013c50e68/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:51.878762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14adab9-02eb-4029-8f5a-ac116f8d8c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4b0502715d43baa39c77f6d0fa46b3, user_id=571506ba11a7489797ae2c2013c50e68, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:51.886577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14adab9-02eb-4029-8f5a-ac116f8d8c37 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:51.901565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7a5bf7e6-70ee-4b0d-a0b5-d4c34c3dde6f None None] MFA Rules not processed for user `92867e15dcf649adad78bf7a29a996f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:51.903607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7a5bf7e6-70ee-4b0d-a0b5-d4c34c3dde6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:51.933338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 846/1705] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:46:51 2022] PUT /identity/v3/projects/7d4b0502715d43baa39c77f6d0fa46b3/users/571506ba11a7489797ae2c2013c50e68/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:52.032006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 860/1706] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:46:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:52.067748 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14adab9-02eb-4029-8f5a-ac116f8d8c37 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:52.075049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d14adab9-02eb-4029-8f5a-ac116f8d8c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:52.139366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14adab9-02eb-4029-8f5a-ac116f8d8c37 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:52.148323 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a5c1c-8381-4fef-9214-116cb80ffc83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:52.153669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a5c1c-8381-4fef-9214-116cb80ffc83 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:52.153669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a5c1c-8381-4fef-9214-116cb80ffc83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:52.153669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a5c1c-8381-4fef-9214-116cb80ffc83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:52.168692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a5c1c-8381-4fef-9214-116cb80ffc83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:52.170170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a5c1c-8381-4fef-9214-116cb80ffc83 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:52.192984 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 847/1707] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:52.292007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:52.320622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bf7e6-70ee-4b0d-a0b5-d4c34c3dde6f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:52.438498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bf7e6-70ee-4b0d-a0b5-d4c34c3dde6f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:52.448041 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eabb67-1605-46e4-9011-fd299370e231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:52.449367 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eabb67-1605-46e4-9011-fd299370e231 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:52.452376 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eabb67-1605-46e4-9011-fd299370e231 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:52.452376 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eabb67-1605-46e4-9011-fd299370e231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:52.688565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eabb67-1605-46e4-9011-fd299370e231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:52.693121 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eabb67-1605-46e4-9011-fd299370e231 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:52.723160 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 861/1708] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:53.422627 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a5c1c-8381-4fef-9214-116cb80ffc83 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:53.424964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-377a5c1c-8381-4fef-9214-116cb80ffc83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:53.457225 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:53.744702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a5c1c-8381-4fef-9214-116cb80ffc83 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:53.763394 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e87a85-3ecc-42bf-9b88-cf4e4a6e9a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:53.765425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e87a85-3ecc-42bf-9b88-cf4e4a6e9a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:53.771458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e87a85-3ecc-42bf-9b88-cf4e4a6e9a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:53.774398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e87a85-3ecc-42bf-9b88-cf4e4a6e9a48 admin admin] PATH_INFO: `/v3/users/1a0d8dda074b4f1da9f6c12dffa9b169` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:53.777993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e87a85-3ecc-42bf-9b88-cf4e4a6e9a48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a0d8dda074b4f1da9f6c12dffa9b169)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:53.806107 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e87a85-3ecc-42bf-9b88-cf4e4a6e9a48 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:53.875269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eabb67-1605-46e4-9011-fd299370e231 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:53.877455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-38eabb67-1605-46e4-9011-fd299370e231 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:54.110771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3e87a85-3ecc-42bf-9b88-cf4e4a6e9a48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a0d8dda074b4f1da9f6c12dffa9b169'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:54.127258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eabb67-1605-46e4-9011-fd299370e231 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:54.147756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843c319-b7ff-45e0-9f52-fc5551e4a1c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d03db170fb6460794e32669456bcdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:54.148678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843c319-b7ff-45e0-9f52-fc5551e4a1c8 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:54.149130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843c319-b7ff-45e0-9f52-fc5551e4a1c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:54.149597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843c319-b7ff-45e0-9f52-fc5551e4a1c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:54.154680 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d843c319-b7ff-45e0-9f52-fc5551e4a1c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:54.183421 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3e87a85-3ecc-42bf-9b88-cf4e4a6e9a48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a0d8dda074b4f1da9f6c12dffa9b169'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:54.185046 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843c319-b7ff-45e0-9f52-fc5551e4a1c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:54.187335 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843c319-b7ff-45e0-9f52-fc5551e4a1c8 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:54.210749 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 862/1709] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:46:54.211751 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843c319-b7ff-45e0-9f52-fc5551e4a1c8 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:54.223977 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843c319-b7ff-45e0-9f52-fc5551e4a1c8 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:54.229351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21395651-b203-4774-b766-2aa9c8f73891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:54.230217 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21395651-b203-4774-b766-2aa9c8f73891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:54.230721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21395651-b203-4774-b766-2aa9c8f73891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:54.231193 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21395651-b203-4774-b766-2aa9c8f73891 admin admin] PATH_INFO: `/v3/users/549f76f68dfa441095c74bbb6ab02d32` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:54.232980 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21395651-b203-4774-b766-2aa9c8f73891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=549f76f68dfa441095c74bbb6ab02d32)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:54.235359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21395651-b203-4774-b766-2aa9c8f73891 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:54.408185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3e87a85-3ecc-42bf-9b88-cf4e4a6e9a48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a0d8dda074b4f1da9f6c12dffa9b169'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:54.409143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3e87a85-3ecc-42bf-9b88-cf4e4a6e9a48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a0d8dda074b4f1da9f6c12dffa9b169'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:54.429895 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3e87a85-3ecc-42bf-9b88-cf4e4a6e9a48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a0d8dda074b4f1da9f6c12dffa9b169'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:54.447346 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 848/1710] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:46:53 2022] DELETE /identity/v3/users/1a0d8dda074b4f1da9f6c12dffa9b169 => generated 0 bytes in 1026 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:54.455402 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77463198-d81e-4fcd-856d-239783f41cfa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:54.455402 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77463198-d81e-4fcd-856d-239783f41cfa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:54.455402 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77463198-d81e-4fcd-856d-239783f41cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:54.496296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-77463198-d81e-4fcd-856d-239783f41cfa None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:54.500027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-77463198-d81e-4fcd-856d-239783f41cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:54.572196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:54.606705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-21395651-b203-4774-b766-2aa9c8f73891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '549f76f68dfa441095c74bbb6ab02d32'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:54.607792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-21395651-b203-4774-b766-2aa9c8f73891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '549f76f68dfa441095c74bbb6ab02d32'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:54.733114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 849/1711] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:54.741818 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-21395651-b203-4774-b766-2aa9c8f73891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '549f76f68dfa441095c74bbb6ab02d32'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:54.756118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77463198-d81e-4fcd-856d-239783f41cfa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:54.763536 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-77463198-d81e-4fcd-856d-239783f41cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:54.774023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-21395651-b203-4774-b766-2aa9c8f73891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '549f76f68dfa441095c74bbb6ab02d32'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:54.839723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-21395651-b203-4774-b766-2aa9c8f73891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '549f76f68dfa441095c74bbb6ab02d32'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:54.866548 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 863/1712] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:46:54 2022] DELETE /identity/v3/users/549f76f68dfa441095c74bbb6ab02d32 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:46:54.873984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229a863-47c2-4e5d-a6bf-753afcf3a578 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:54.875212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229a863-47c2-4e5d-a6bf-753afcf3a578 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:54.875853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229a863-47c2-4e5d-a6bf-753afcf3a578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:54.975821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:54.988875 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8229a863-47c2-4e5d-a6bf-753afcf3a578 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:54.990232 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8229a863-47c2-4e5d-a6bf-753afcf3a578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:55.157691 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77463198-d81e-4fcd-856d-239783f41cfa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:55.237489 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 864/1713] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:55.250331 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acb2e0-276d-47f0-9ccf-bca35098e4c1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:55.250794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acb2e0-276d-47f0-9ccf-bca35098e4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:55.253239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acb2e0-276d-47f0-9ccf-bca35098e4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:55.358078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-34acb2e0-276d-47f0-9ccf-bca35098e4c1 None None] MFA Rules not processed for user `64ea6fbc6e7d42dabdf459e8de4c3621`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:55.359738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-34acb2e0-276d-47f0-9ccf-bca35098e4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:55.511215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c10657-3628-400d-b65d-6b1849eb7063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:55.511870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c10657-3628-400d-b65d-6b1849eb7063 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:55.512147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c10657-3628-400d-b65d-6b1849eb7063 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:55.512415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c10657-3628-400d-b65d-6b1849eb7063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:55.548508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 865/1714] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:55.552122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acb2e0-276d-47f0-9ccf-bca35098e4c1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:55.558637 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c10657-3628-400d-b65d-6b1849eb7063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:55.558870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-34acb2e0-276d-47f0-9ccf-bca35098e4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:55.565763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c10657-3628-400d-b65d-6b1849eb7063 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:55.601320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 850/1715] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 847 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:55.607132 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c10657-3628-400d-b65d-6b1849eb7063 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:55.610783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acb2e0-276d-47f0-9ccf-bca35098e4c1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:55.615499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88c10657-3628-400d-b65d-6b1849eb7063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:55.627131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9e61a-a4f6-4c84-a5d1-bf69bcb563f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:55.628467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9e61a-a4f6-4c84-a5d1-bf69bcb563f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:55.631521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9e61a-a4f6-4c84-a5d1-bf69bcb563f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:55.631796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9e61a-a4f6-4c84-a5d1-bf69bcb563f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:55.669480 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9e61a-a4f6-4c84-a5d1-bf69bcb563f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:55.670967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9e61a-a4f6-4c84-a5d1-bf69bcb563f2 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:55.711620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 866/1716] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:55.766750 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c10657-3628-400d-b65d-6b1849eb7063 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:55.777324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14795812-0c47-4b4f-b150-57930a4aadb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:55.777324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14795812-0c47-4b4f-b150-57930a4aadb5 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:55.777324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14795812-0c47-4b4f-b150-57930a4aadb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:55.777324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14795812-0c47-4b4f-b150-57930a4aadb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:55.786041 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14795812-0c47-4b4f-b150-57930a4aadb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:55.788150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14795812-0c47-4b4f-b150-57930a4aadb5 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:55.802056 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 851/1717] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:56.262659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeca73-6e38-4182-8d7a-0226f0c0a40d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:56.264118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeca73-6e38-4182-8d7a-0226f0c0a40d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:56.264981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eeca73-6e38-4182-8d7a-0226f0c0a40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:56.367261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b1eeca73-6e38-4182-8d7a-0226f0c0a40d None None] MFA Rules not processed for user `2aea4a0e0ec74c538735b890fb774b14`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:56.371440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b1eeca73-6e38-4182-8d7a-0226f0c0a40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:56.392973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:56.574731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 867/1718] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:46:56.596204 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84272b9a-a3fa-427f-b77b-f47ac84ec64d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:56.596888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84272b9a-a3fa-427f-b77b-f47ac84ec64d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:56.597272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84272b9a-a3fa-427f-b77b-f47ac84ec64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:56.674650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-84272b9a-a3fa-427f-b77b-f47ac84ec64d None None] MFA Rules not processed for user `2aea4a0e0ec74c538735b890fb774b14`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:56.680405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-84272b9a-a3fa-427f-b77b-f47ac84ec64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:56.702586 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 852/1719] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:56.714874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eeca73-6e38-4182-8d7a-0226f0c0a40d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:56.717205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b1eeca73-6e38-4182-8d7a-0226f0c0a40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:56.783613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eeca73-6e38-4182-8d7a-0226f0c0a40d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:56.807667 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d88771-0555-4384-87f4-d2c04226a55b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:56.809269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d88771-0555-4384-87f4-d2c04226a55b service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:56.809832 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d88771-0555-4384-87f4-d2c04226a55b service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:56.814771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d88771-0555-4384-87f4-d2c04226a55b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:56.870233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d88771-0555-4384-87f4-d2c04226a55b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:56.888103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d88771-0555-4384-87f4-d2c04226a55b service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:56.948909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 868/1720] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:46:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:46:57.272059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84272b9a-a3fa-427f-b77b-f47ac84ec64d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:57.274444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-84272b9a-a3fa-427f-b77b-f47ac84ec64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:57.326653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84272b9a-a3fa-427f-b77b-f47ac84ec64d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:57.344255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:57.346140 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:57.347107 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:57.347842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] PATH_INFO: `/v3/projects/b13af905788d42c39b3233055a34589d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:57.351239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b13af905788d42c39b3233055a34589d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:57.361279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:57.573879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dabe4-8fce-4f2e-9fcb-60b4a5eb674f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:57.575696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dabe4-8fce-4f2e-9fcb-60b4a5eb674f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:57.576606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dabe4-8fce-4f2e-9fcb-60b4a5eb674f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:57.640665 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b13af905788d42c39b3233055a34589d', 'request_id': 'req-10772f62-db49-4eb5-b501-11edb2c12097'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:57.669856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b86dabe4-8fce-4f2e-9fcb-60b4a5eb674f None None] MFA Rules not processed for user `571506ba11a7489797ae2c2013c50e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:46:57.672259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b86dabe4-8fce-4f2e-9fcb-60b4a5eb674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:57.750927 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b13af905788d42c39b3233055a34589d', 'request_id': 'req-10772f62-db49-4eb5-b501-11edb2c12097'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:57.753271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:46:57.773945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b13af905788d42c39b3233055a34589d', 'request_id': 'req-10772f62-db49-4eb5-b501-11edb2c12097'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:57.820815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] The token cache is being invalidate because project b13af905788d42c39b3233055a34589d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:46:57.822414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:57.826441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:57.861415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 853/1721] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:46:57 2022] DELETE /identity/v3/projects/b13af905788d42c39b3233055a34589d => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:46:57.863105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:57.866138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:57.922779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 869/1722] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:46:57 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:46:58.200130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10772f62-db49-4eb5-b501-11edb2c12097 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:46:58.211751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b428db-9830-4797-9ce6-78f4b44f514e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:46:58.214981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b428db-9830-4797-9ce6-78f4b44f514e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:46:58.215840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b428db-9830-4797-9ce6-78f4b44f514e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:46:58.216896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b428db-9830-4797-9ce6-78f4b44f514e admin admin] PATH_INFO: `/v3/users/4eaade9548fb48a085f29dc57ffbe49e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:46:58.219730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b428db-9830-4797-9ce6-78f4b44f514e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eaade9548fb48a085f29dc57ffbe49e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:46:58.224050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b428db-9830-4797-9ce6-78f4b44f514e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:46:58.667771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c1b428db-9830-4797-9ce6-78f4b44f514e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eaade9548fb48a085f29dc57ffbe49e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:58.814165 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c1b428db-9830-4797-9ce6-78f4b44f514e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eaade9548fb48a085f29dc57ffbe49e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:58.985790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c1b428db-9830-4797-9ce6-78f4b44f514e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eaade9548fb48a085f29dc57ffbe49e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:58.986738 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c1b428db-9830-4797-9ce6-78f4b44f514e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eaade9548fb48a085f29dc57ffbe49e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:59.016813 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c1b428db-9830-4797-9ce6-78f4b44f514e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eaade9548fb48a085f29dc57ffbe49e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:46:59.044600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 854/1723] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:46:57 2022] DELETE /identity/v3/users/4eaade9548fb48a085f29dc57ffbe49e => generated 0 bytes in 1182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:46:59.810062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dabe4-8fce-4f2e-9fcb-60b4a5eb674f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:46:59.812998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b86dabe4-8fce-4f2e-9fcb-60b4a5eb674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:46:59.910356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dabe4-8fce-4f2e-9fcb-60b4a5eb674f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:00.165092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97a0be-a209-4588-9729-cb11e1339eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:00.166600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97a0be-a209-4588-9729-cb11e1339eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:00.167186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97a0be-a209-4588-9729-cb11e1339eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:00.167707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97a0be-a209-4588-9729-cb11e1339eb5 admin admin] PATH_INFO: `/v3/projects/aa8606e30b454f6d948af7cbcb809f34` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:00.179789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97a0be-a209-4588-9729-cb11e1339eb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8606e30b454f6d948af7cbcb809f34)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:00.187459 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97a0be-a209-4588-9729-cb11e1339eb5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:00.335792 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b258ce-5d39-4233-97a6-f4d73c9d8926 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:00.336275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b258ce-5d39-4233-97a6-f4d73c9d8926 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:00.336591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b258ce-5d39-4233-97a6-f4d73c9d8926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:00.383894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:00.425696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e3b258ce-5d39-4233-97a6-f4d73c9d8926 None None] MFA Rules not processed for user `3abba619decf448db477533c663004bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:00.427763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e3b258ce-5d39-4233-97a6-f4d73c9d8926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:00.470351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ac97a0be-a209-4588-9729-cb11e1339eb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8606e30b454f6d948af7cbcb809f34', 'request_id': 'req-ac97a0be-a209-4588-9729-cb11e1339eb5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:00.580156 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:00.602659 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 855/1724] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:47:00.612108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ac97a0be-a209-4588-9729-cb11e1339eb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8606e30b454f6d948af7cbcb809f34', 'request_id': 'req-ac97a0be-a209-4588-9729-cb11e1339eb5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:00.639964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ac97a0be-a209-4588-9729-cb11e1339eb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8606e30b454f6d948af7cbcb809f34', 'request_id': 'req-ac97a0be-a209-4588-9729-cb11e1339eb5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:00.677402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ac97a0be-a209-4588-9729-cb11e1339eb5 admin admin] The token cache is being invalidate because project aa8606e30b454f6d948af7cbcb809f34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:00.677402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ac97a0be-a209-4588-9729-cb11e1339eb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:00.678934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ac97a0be-a209-4588-9729-cb11e1339eb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:00.682260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 870/1725] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:46:59 2022] DELETE /identity/v3/projects/aa8606e30b454f6d948af7cbcb809f34 => generated 0 bytes in 873 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:00.691053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4be718-6004-4eff-b1bb-8214026c5cdd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:00.691545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4be718-6004-4eff-b1bb-8214026c5cdd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:00.692038 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4be718-6004-4eff-b1bb-8214026c5cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:00.728763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6a4be718-6004-4eff-b1bb-8214026c5cdd None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:00.731129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6a4be718-6004-4eff-b1bb-8214026c5cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:00.857783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 856/1726] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:00.873734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1536730-496b-48cc-bb7c-72a693c3d5f8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:00.874276 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1536730-496b-48cc-bb7c-72a693c3d5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:00.874650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1536730-496b-48cc-bb7c-72a693c3d5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:00.913638 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4be718-6004-4eff-b1bb-8214026c5cdd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:00.923481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6a4be718-6004-4eff-b1bb-8214026c5cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:01.054722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f1536730-496b-48cc-bb7c-72a693c3d5f8 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:01.060780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f1536730-496b-48cc-bb7c-72a693c3d5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:01.128926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 871/1727] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:01.136870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1536730-496b-48cc-bb7c-72a693c3d5f8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:01.190810 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1536730-496b-48cc-bb7c-72a693c3d5f8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:01.204122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0ca2d-e078-4c8d-a03f-c9d09877f028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:01.205344 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0ca2d-e078-4c8d-a03f-c9d09877f028 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:01.205700 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0ca2d-e078-4c8d-a03f-c9d09877f028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:01.206343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0ca2d-e078-4c8d-a03f-c9d09877f028 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:01.207861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0ca2d-e078-4c8d-a03f-c9d09877f028 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:47:01.208361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0ca2d-e078-4c8d-a03f-c9d09877f028 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:01.211765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0ca2d-e078-4c8d-a03f-c9d09877f028 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:01.278927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 872/1728] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:47:01.285784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0ca2d-e078-4c8d-a03f-c9d09877f028 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:01.347148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0ca2d-e078-4c8d-a03f-c9d09877f028 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:01.368429 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817658d6-67c8-4b72-83bd-1ba4966f0f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:01.369504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817658d6-67c8-4b72-83bd-1ba4966f0f2d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:01.374827 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817658d6-67c8-4b72-83bd-1ba4966f0f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:01.374827 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817658d6-67c8-4b72-83bd-1ba4966f0f2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:01.374827 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817658d6-67c8-4b72-83bd-1ba4966f0f2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:01.388665 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817658d6-67c8-4b72-83bd-1ba4966f0f2d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:01.392258 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4be718-6004-4eff-b1bb-8214026c5cdd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:01.420543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2831f-2bfa-43c1-9d9b-e3a5d87a5d03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:01.422518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2831f-2bfa-43c1-9d9b-e3a5d87a5d03 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:01.426541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2831f-2bfa-43c1-9d9b-e3a5d87a5d03 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:01.428841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2831f-2bfa-43c1-9d9b-e3a5d87a5d03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:01.435591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5fb2831f-2bfa-43c1-9d9b-e3a5d87a5d03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:01.511534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2831f-2bfa-43c1-9d9b-e3a5d87a5d03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:01.514097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2831f-2bfa-43c1-9d9b-e3a5d87a5d03 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:01.551858 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 873/1729] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:01 2022] POST /identity/v3/projects => generated 383 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:01.555005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817658d6-67c8-4b72-83bd-1ba4966f0f2d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:01.565788 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 857/1730] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:47:01.764441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817658d6-67c8-4b72-83bd-1ba4966f0f2d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:01.766916 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c2d28-2a84-42d9-9ad6-53ae8d506b4a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:01.767204 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c2d28-2a84-42d9-9ad6-53ae8d506b4a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:01.767507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c2d28-2a84-42d9-9ad6-53ae8d506b4a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:01.769546 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 858/1731] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:47:01.783869 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ca4c6-7751-40e8-86a1-27ccbe4369e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:01.784599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ca4c6-7751-40e8-86a1-27ccbe4369e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:01.784988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ca4c6-7751-40e8-86a1-27ccbe4369e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:01.785269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ca4c6-7751-40e8-86a1-27ccbe4369e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:01.786025 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ca4c6-7751-40e8-86a1-27ccbe4369e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:01.790054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ca4c6-7751-40e8-86a1-27ccbe4369e4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:01.902076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 874/1732] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:01 2022] POST /identity/v3/users => generated 292 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:01.903391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c2d28-2a84-42d9-9ad6-53ae8d506b4a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:01.921518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def268b-9486-45af-958d-c7a4a798c682 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:01.922292 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def268b-9486-45af-958d-c7a4a798c682 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:01.922573 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def268b-9486-45af-958d-c7a4a798c682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:01.951888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c2d28-2a84-42d9-9ad6-53ae8d506b4a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:01.970289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8bac0-4aa4-4b70-8311-f129bdbe8fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:01.970881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8bac0-4aa4-4b70-8311-f129bdbe8fae admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:01.971197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8bac0-4aa4-4b70-8311-f129bdbe8fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:01.971473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8bac0-4aa4-4b70-8311-f129bdbe8fae admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:01.972122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8bac0-4aa4-4b70-8311-f129bdbe8fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:01.973841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8bac0-4aa4-4b70-8311-f129bdbe8fae admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:02.023688 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7def268b-9486-45af-958d-c7a4a798c682 None None] MFA Rules not processed for user `8d03db170fb6460794e32669456bcdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:02.031482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 859/1733] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:02.033003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8bac0-4aa4-4b70-8311-f129bdbe8fae admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:02.033936 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7def268b-9486-45af-958d-c7a4a798c682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:02.064257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8bac0-4aa4-4b70-8311-f129bdbe8fae admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:02.076533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ece1d1-ec2d-4a0f-8c55-7436b49b3d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:02.077549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ece1d1-ec2d-4a0f-8c55-7436b49b3d26 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:02.078116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ece1d1-ec2d-4a0f-8c55-7436b49b3d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:02.078673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ece1d1-ec2d-4a0f-8c55-7436b49b3d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:02.079544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ece1d1-ec2d-4a0f-8c55-7436b49b3d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:02.087413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ece1d1-ec2d-4a0f-8c55-7436b49b3d26 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:02.116944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 860/1734] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:02.118106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ece1d1-ec2d-4a0f-8c55-7436b49b3d26 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:02.149617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ece1d1-ec2d-4a0f-8c55-7436b49b3d26 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:02.157216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e57ef-0ef5-4186-be24-4b1bc659176f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:02.158729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e57ef-0ef5-4186-be24-4b1bc659176f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:02.159406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e57ef-0ef5-4186-be24-4b1bc659176f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:02.160429 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e57ef-0ef5-4186-be24-4b1bc659176f admin admin] PATH_INFO: `/v3/projects/c748ed5480cd417ea9addcc719399b46/users/73a56f8d4421480ea0037459173f709d/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:02.207351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e57ef-0ef5-4186-be24-4b1bc659176f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c748ed5480cd417ea9addcc719399b46, user_id=73a56f8d4421480ea0037459173f709d, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:02.213049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e57ef-0ef5-4186-be24-4b1bc659176f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:02.343882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 861/1735] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:02 2022] PUT /identity/v3/projects/c748ed5480cd417ea9addcc719399b46/users/73a56f8d4421480ea0037459173f709d/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:02.345317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e57ef-0ef5-4186-be24-4b1bc659176f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:02.351064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 875/1736] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:47:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:02.351392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dc0e57ef-0ef5-4186-be24-4b1bc659176f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:02.380375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bd1a9-4218-47b5-8609-10b753f557f6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:02.384572 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bd1a9-4218-47b5-8609-10b753f557f6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:02.385825 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bd1a9-4218-47b5-8609-10b753f557f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:02.409803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:02.472062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ed5bd1a9-4218-47b5-8609-10b753f557f6 None None] MFA Rules not processed for user `2aea4a0e0ec74c538735b890fb774b14`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:02.476870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ed5bd1a9-4218-47b5-8609-10b753f557f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:02.524640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:02.598171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e57ef-0ef5-4186-be24-4b1bc659176f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:02.605442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:02.606279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:02.606616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:02.606929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] PATH_INFO: `/v3/projects/efaaa4f11d664a09a81ddd728c564f73` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:02.608871 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efaaa4f11d664a09a81ddd728c564f73)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:02.613054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:02.797028 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 876/1737] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:02.799733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b0286-83fc-41ef-9781-432bd4517021 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:02.800397 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b0286-83fc-41ef-9781-432bd4517021 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:02.843494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b0286-83fc-41ef-9781-432bd4517021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:02.950997 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efaaa4f11d664a09a81ddd728c564f73', 'request_id': 'req-b9941578-db93-4b86-b9c9-6716c3d28724'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:03.025820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f59b0286-83fc-41ef-9781-432bd4517021 None None] MFA Rules not processed for user `73a56f8d4421480ea0037459173f709d`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:03.030480 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f59b0286-83fc-41ef-9781-432bd4517021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:03.262460 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efaaa4f11d664a09a81ddd728c564f73', 'request_id': 'req-b9941578-db93-4b86-b9c9-6716c3d28724'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:03.296926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 877/1738] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:03.298654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819a758-4552-404d-878b-b2bb668991ac None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:03.300300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819a758-4552-404d-878b-b2bb668991ac None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:03.300750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819a758-4552-404d-878b-b2bb668991ac None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:03.304778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 878/1739] 10.222.0.44 () {54 vars in 961 bytes} [Sun Sep 11 08:47:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:47:03.308732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c671-ced5-4b80-a77d-93e4dc9988d5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:03.309222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c671-ced5-4b80-a77d-93e4dc9988d5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:03.309641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c671-ced5-4b80-a77d-93e4dc9988d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:03.320007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efaaa4f11d664a09a81ddd728c564f73', 'request_id': 'req-b9941578-db93-4b86-b9c9-6716c3d28724'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:03.395926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] The token cache is being invalidate because project efaaa4f11d664a09a81ddd728c564f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:03.399556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:03.400860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:03.407280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 862/1740] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:02 2022] DELETE /identity/v3/projects/efaaa4f11d664a09a81ddd728c564f73 => generated 0 bytes in 1062 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:03.408766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:03.413341 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:03.472376 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ee68c671-ced5-4b80-a77d-93e4dc9988d5 None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:03.473916 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ee68c671-ced5-4b80-a77d-93e4dc9988d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:03.673922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9941578-db93-4b86-b9c9-6716c3d28724 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:03.700316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96040f87-4321-4013-84a4-03125503d8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:03.704183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96040f87-4321-4013-84a4-03125503d8c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:03.705017 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96040f87-4321-4013-84a4-03125503d8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:03.706399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96040f87-4321-4013-84a4-03125503d8c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:03.710180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96040f87-4321-4013-84a4-03125503d8c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:03.713674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96040f87-4321-4013-84a4-03125503d8c1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:03.756169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 879/1741] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:47:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:03.759760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897983a3-3a5f-4c2e-80af-26683ca836d3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:03.764390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897983a3-3a5f-4c2e-80af-26683ca836d3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:03.764841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897983a3-3a5f-4c2e-80af-26683ca836d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:03.822856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-897983a3-3a5f-4c2e-80af-26683ca836d3 None None] MFA Rules not processed for user `8d03db170fb6460794e32669456bcdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:03.830371 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-897983a3-3a5f-4c2e-80af-26683ca836d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:03.842965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:03.898456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 863/1742] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:03 2022] POST /identity/v3/projects => generated 383 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:03.899864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebeb6c-eadf-4e0a-a67f-1639af153222 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:03.900136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebeb6c-eadf-4e0a-a67f-1639af153222 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:03.900414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebeb6c-eadf-4e0a-a67f-1639af153222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:03.996859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-55ebeb6c-eadf-4e0a-a67f-1639af153222 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:04.007531 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-55ebeb6c-eadf-4e0a-a67f-1639af153222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:04.082644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 880/1743] 10.222.0.44 () {58 vars in 1058 bytes} [Sun Sep 11 08:47:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:04.082644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897983a3-3a5f-4c2e-80af-26683ca836d3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:04.111447 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897983a3-3a5f-4c2e-80af-26683ca836d3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:04.225562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:04.307684 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 864/1744] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:04.309452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ebeb6c-eadf-4e0a-a67f-1639af153222 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:04.400674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f985a1-7457-486a-9690-17fccc52ae96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:04.401480 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f985a1-7457-486a-9690-17fccc52ae96 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:04.401758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f985a1-7457-486a-9690-17fccc52ae96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:04.402035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f985a1-7457-486a-9690-17fccc52ae96 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:04.402738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f985a1-7457-486a-9690-17fccc52ae96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:04.407032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f985a1-7457-486a-9690-17fccc52ae96 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:04.546568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 881/1745] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:04 2022] POST /identity/v3/users => generated 298 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:04.546568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a4777-1e74-4917-85a2-172e20a797e5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:04.546568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a4777-1e74-4917-85a2-172e20a797e5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:04.546568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a4777-1e74-4917-85a2-172e20a797e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:04.608833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ebeb6c-eadf-4e0a-a67f-1639af153222 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:04.642293 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12517843-1e79-452f-a3e1-2d9cea0b67f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d03db170fb6460794e32669456bcdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:04.643800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12517843-1e79-452f-a3e1-2d9cea0b67f8 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:04.650091 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12517843-1e79-452f-a3e1-2d9cea0b67f8 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:04.650516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-920a4777-1e74-4917-85a2-172e20a797e5 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:04.650701 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12517843-1e79-452f-a3e1-2d9cea0b67f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:04.656078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-920a4777-1e74-4917-85a2-172e20a797e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:04.664583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-12517843-1e79-452f-a3e1-2d9cea0b67f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:04.705002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 882/1746] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:04.707327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a4777-1e74-4917-85a2-172e20a797e5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:04.711808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-920a4777-1e74-4917-85a2-172e20a797e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:04.713207 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12517843-1e79-452f-a3e1-2d9cea0b67f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:04.716842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12517843-1e79-452f-a3e1-2d9cea0b67f8 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:04.757457 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 865/1747] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:04.759166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f401530-28df-49da-80e9-06bca1c91b05 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:04.759467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f401530-28df-49da-80e9-06bca1c91b05 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:04.759710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f401530-28df-49da-80e9-06bca1c91b05 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:04.761205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 866/1748] 10.222.0.44 () {54 vars in 912 bytes} [Sun Sep 11 08:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:47:04.768766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f401530-28df-49da-80e9-06bca1c91b05 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:04.801899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f401530-28df-49da-80e9-06bca1c91b05 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:04.814316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d3080-c4af-49bc-beb3-cff9fa63ffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:04.814316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d3080-c4af-49bc-beb3-cff9fa63ffbd admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:04.814316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d3080-c4af-49bc-beb3-cff9fa63ffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:04.814316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d3080-c4af-49bc-beb3-cff9fa63ffbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:04.815919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d3080-c4af-49bc-beb3-cff9fa63ffbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:04.815919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d3080-c4af-49bc-beb3-cff9fa63ffbd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:04.844064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 867/1749] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:04.844064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d3080-c4af-49bc-beb3-cff9fa63ffbd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:04.872342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d3080-c4af-49bc-beb3-cff9fa63ffbd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:04.883237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901d48c-4324-498c-8433-7ab12154a0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:04.883237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901d48c-4324-498c-8433-7ab12154a0fe admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:04.883237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901d48c-4324-498c-8433-7ab12154a0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:04.883237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901d48c-4324-498c-8433-7ab12154a0fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:04.883773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901d48c-4324-498c-8433-7ab12154a0fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:47:04.883773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901d48c-4324-498c-8433-7ab12154a0fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:04.885015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901d48c-4324-498c-8433-7ab12154a0fe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:04.921929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 868/1750] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:47:04.924009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f8e49-276b-4d62-adc5-6a1b0ba821bf None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:04.924009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f8e49-276b-4d62-adc5-6a1b0ba821bf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:04.924009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f8e49-276b-4d62-adc5-6a1b0ba821bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:04.971623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-510f8e49-276b-4d62-adc5-6a1b0ba821bf None None] MFA Rules not processed for user `92867e15dcf649adad78bf7a29a996f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:04.972951 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-510f8e49-276b-4d62-adc5-6a1b0ba821bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:05.061636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a4777-1e74-4917-85a2-172e20a797e5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:05.068044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8c3c8-270d-4482-bdc2-5dae20d3d582 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:05.068555 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8c3c8-270d-4482-bdc2-5dae20d3d582 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:05.068859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8c3c8-270d-4482-bdc2-5dae20d3d582 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:05.069144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8c3c8-270d-4482-bdc2-5dae20d3d582 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:05.083207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:05.131491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 869/1751] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:47:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:05.132106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f8e49-276b-4d62-adc5-6a1b0ba821bf None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:05.161313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f8e49-276b-4d62-adc5-6a1b0ba821bf None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:05.191281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fd31d-0351-4d47-8c62-a517ae61a4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:05.191987 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd31d-0351-4d47-8c62-a517ae61a4b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:05.192266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd31d-0351-4d47-8c62-a517ae61a4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:05.192539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd31d-0351-4d47-8c62-a517ae61a4b2 admin admin] PATH_INFO: `/v3/projects/321a2870a48d4309836fd2e5c88169f0/users/0c0d740c37414c81a9f045c545eaa54e/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:05.224808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8c3c8-270d-4482-bdc2-5dae20d3d582 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:05.226555 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8c3c8-270d-4482-bdc2-5dae20d3d582 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:05.238840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6fd31d-0351-4d47-8c62-a517ae61a4b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=321a2870a48d4309836fd2e5c88169f0, user_id=0c0d740c37414c81a9f045c545eaa54e, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:05.241652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6fd31d-0351-4d47-8c62-a517ae61a4b2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:05.260007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 883/1752] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:47:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:47:05.261441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8c3c8-270d-4482-bdc2-5dae20d3d582 service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:05.286128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8c3c8-270d-4482-bdc2-5dae20d3d582 service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:05.301923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6b3dc-396f-447d-8e99-15e9b35f22f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:05.303516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6b3dc-396f-447d-8e99-15e9b35f22f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:05.311202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6b3dc-396f-447d-8e99-15e9b35f22f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:05.312139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6b3dc-396f-447d-8e99-15e9b35f22f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:05.324631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6b3dc-396f-447d-8e99-15e9b35f22f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:05.341760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6b3dc-396f-447d-8e99-15e9b35f22f5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:05.358497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 870/1753] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:05 2022] PUT /identity/v3/projects/321a2870a48d4309836fd2e5c88169f0/users/0c0d740c37414c81a9f045c545eaa54e/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:05.364685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fd31d-0351-4d47-8c62-a517ae61a4b2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:05.556700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:05.660418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 884/1754] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:05 2022] POST /identity/v3/projects => generated 367 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:05.660418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6b3dc-396f-447d-8e99-15e9b35f22f5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:05.665563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fd31d-0351-4d47-8c62-a517ae61a4b2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:05.679307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-35e6b3dc-396f-447d-8e99-15e9b35f22f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:05.898330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f8451-191a-4001-958a-4d1623ca305a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:05.899591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f8451-191a-4001-958a-4d1623ca305a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:05.900819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f8451-191a-4001-958a-4d1623ca305a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:05.902304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f8451-191a-4001-958a-4d1623ca305a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:05.903028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f8451-191a-4001-958a-4d1623ca305a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:05.905028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f8451-191a-4001-958a-4d1623ca305a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:05.922886 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 871/1755] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 558 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:47:05.924033 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f8451-191a-4001-958a-4d1623ca305a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:05.942013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f8451-191a-4001-958a-4d1623ca305a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:05.951815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d2209-2b1f-41a9-940a-fb464e53a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:05.956340 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2209-2b1f-41a9-940a-fb464e53a85a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:05.959847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2209-2b1f-41a9-940a-fb464e53a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:05.962493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2209-2b1f-41a9-940a-fb464e53a85a admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:05.966471 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d2209-2b1f-41a9-940a-fb464e53a85a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:05.975370 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d2209-2b1f-41a9-940a-fb464e53a85a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:06.029288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6b3dc-396f-447d-8e99-15e9b35f22f5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:06.042637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e682842-1aa7-404e-9cb8-81cbe4d6e6d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:06.043994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e682842-1aa7-404e-9cb8-81cbe4d6e6d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:06.044606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e682842-1aa7-404e-9cb8-81cbe4d6e6d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:06.045278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e682842-1aa7-404e-9cb8-81cbe4d6e6d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:06.115132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e682842-1aa7-404e-9cb8-81cbe4d6e6d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:06.124946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e682842-1aa7-404e-9cb8-81cbe4d6e6d1 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:06.143048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 872/1756] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:05 2022] POST /identity/v3/users => generated 284 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:06.144384 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d2209-2b1f-41a9-940a-fb464e53a85a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:06.198533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 885/1757] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:06.203686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e682842-1aa7-404e-9cb8-81cbe4d6e6d1 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:06.221797 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d2209-2b1f-41a9-940a-fb464e53a85a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:06.235293 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad746e8-04ca-41bd-95a0-ef09d241fa1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:06.236004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad746e8-04ca-41bd-95a0-ef09d241fa1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:06.236286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad746e8-04ca-41bd-95a0-ef09d241fa1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:06.238420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad746e8-04ca-41bd-95a0-ef09d241fa1f admin admin] PATH_INFO: `/v3/projects/321a2870a48d4309836fd2e5c88169f0/users/0c0d740c37414c81a9f045c545eaa54e/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:06.245678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad746e8-04ca-41bd-95a0-ef09d241fa1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=321a2870a48d4309836fd2e5c88169f0, user_id=0c0d740c37414c81a9f045c545eaa54e, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:06.252319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad746e8-04ca-41bd-95a0-ef09d241fa1f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:06.258111 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e682842-1aa7-404e-9cb8-81cbe4d6e6d1 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:06.303378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4555be9-42f9-42be-82c6-2c96697331ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:06.306742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4555be9-42f9-42be-82c6-2c96697331ba admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:06.306830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 873/1758] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:06 2022] PUT /identity/v3/projects/321a2870a48d4309836fd2e5c88169f0/users/0c0d740c37414c81a9f045c545eaa54e/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:06.307097 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4555be9-42f9-42be-82c6-2c96697331ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:06.307619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4555be9-42f9-42be-82c6-2c96697331ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:06.308183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64420a8e-5955-4532-85bb-988ed440df69 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:06.310272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64420a8e-5955-4532-85bb-988ed440df69 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:06.310759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64420a8e-5955-4532-85bb-988ed440df69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:06.314523 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4555be9-42f9-42be-82c6-2c96697331ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:06.336113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4555be9-42f9-42be-82c6-2c96697331ba admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:06.406344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-64420a8e-5955-4532-85bb-988ed440df69 None None] MFA Rules not processed for user `0c0d740c37414c81a9f045c545eaa54e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:06.407816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-64420a8e-5955-4532-85bb-988ed440df69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:06.451503 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 886/1759] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:06.464244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b3de6-48b5-46d5-a48b-b901c1a21512 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:06.464750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b3de6-48b5-46d5-a48b-b901c1a21512 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:06.465238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b3de6-48b5-46d5-a48b-b901c1a21512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:06.604273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8a5b3de6-48b5-46d5-a48b-b901c1a21512 None None] MFA Rules not processed for user `64ea6fbc6e7d42dabdf459e8de4c3621`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:06.614967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8a5b3de6-48b5-46d5-a48b-b901c1a21512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:06.617892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 874/1760] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:06.625776 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64420a8e-5955-4532-85bb-988ed440df69 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:06.787817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:06.864609 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64420a8e-5955-4532-85bb-988ed440df69 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:06.884747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e0113-8436-42ff-a03a-1df2eda92d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:06.885786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e0113-8436-42ff-a03a-1df2eda92d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:06.886648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e0113-8436-42ff-a03a-1df2eda92d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:06.887454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e0113-8436-42ff-a03a-1df2eda92d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:06.889942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e0113-8436-42ff-a03a-1df2eda92d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:06.897321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e0113-8436-42ff-a03a-1df2eda92d4c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:06.944614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 875/1761] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:06.946325 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e0113-8436-42ff-a03a-1df2eda92d4c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:06.962289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-389e0113-8436-42ff-a03a-1df2eda92d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:06.988199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 887/1762] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 536 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:06.990327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b3de6-48b5-46d5-a48b-b901c1a21512 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:07.032669 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b3de6-48b5-46d5-a48b-b901c1a21512 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:07.038927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856d29c-874c-49cb-8bdf-d5ddd7399646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:07.039901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856d29c-874c-49cb-8bdf-d5ddd7399646 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:07.040199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856d29c-874c-49cb-8bdf-d5ddd7399646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:07.040466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856d29c-874c-49cb-8bdf-d5ddd7399646 admin admin] PATH_INFO: `/v3/projects/45c18c9452394e9fbdfc1dc38e714f86/users/9e9de6cc5a6c4e46801665eb7109fc77/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:07.067436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856d29c-874c-49cb-8bdf-d5ddd7399646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c18c9452394e9fbdfc1dc38e714f86, user_id=9e9de6cc5a6c4e46801665eb7109fc77, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:07.071371 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856d29c-874c-49cb-8bdf-d5ddd7399646 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:07.181393 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:07.189474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 888/1763] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:06 2022] PUT /identity/v3/projects/45c18c9452394e9fbdfc1dc38e714f86/users/9e9de6cc5a6c4e46801665eb7109fc77/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:07.201353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14f2b-8035-48ca-8174-622d3a5ca632 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:07.201986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14f2b-8035-48ca-8174-622d3a5ca632 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:07.202562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14f2b-8035-48ca-8174-622d3a5ca632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:07.453671 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e0113-8436-42ff-a03a-1df2eda92d4c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:07.457909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-caf14f2b-8035-48ca-8174-622d3a5ca632 None None] MFA Rules not processed for user `9e9de6cc5a6c4e46801665eb7109fc77`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:07.457909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-caf14f2b-8035-48ca-8174-622d3a5ca632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:07.462956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e672f81-331a-4d4d-9abf-58befd18a804 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:07.470410 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e672f81-331a-4d4d-9abf-58befd18a804 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:07.474849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e672f81-331a-4d4d-9abf-58befd18a804 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:07.478944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e672f81-331a-4d4d-9abf-58befd18a804 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:07.582276 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 889/1764] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:07.583784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf14f2b-8035-48ca-8174-622d3a5ca632 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:07.809699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e672f81-331a-4d4d-9abf-58befd18a804 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:07.811352 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e672f81-331a-4d4d-9abf-58befd18a804 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:07.845080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 876/1765] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 899 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:07.978719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf14f2b-8035-48ca-8174-622d3a5ca632 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:08.001427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42455762-912c-4c27-98f5-feb92b3d16ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:08.002671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42455762-912c-4c27-98f5-feb92b3d16ed admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:08.003143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42455762-912c-4c27-98f5-feb92b3d16ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:08.003655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42455762-912c-4c27-98f5-feb92b3d16ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:08.004886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42455762-912c-4c27-98f5-feb92b3d16ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:08.013432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42455762-912c-4c27-98f5-feb92b3d16ed admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:08.115298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50008-9cf7-4b75-adc9-7fe47b9bbfdd None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:08.116179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50008-9cf7-4b75-adc9-7fe47b9bbfdd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:08.116179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50008-9cf7-4b75-adc9-7fe47b9bbfdd None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:08.120686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 877/1766] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:47:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:47:08.248732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 890/1767] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:07 2022] POST /identity/v3/projects => generated 367 bytes in 666 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:08.249710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50008-9cf7-4b75-adc9-7fe47b9bbfdd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:08.267474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42455762-912c-4c27-98f5-feb92b3d16ed admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:08.426614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:08.506944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50008-9cf7-4b75-adc9-7fe47b9bbfdd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:08.532117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4bec2-10bb-4783-96b3-6c2681db6964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:08.537511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4bec2-10bb-4783-96b3-6c2681db6964 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:08.537959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4bec2-10bb-4783-96b3-6c2681db6964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:08.538660 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4bec2-10bb-4783-96b3-6c2681db6964 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:08.539405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4bec2-10bb-4783-96b3-6c2681db6964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:08.550132 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4bec2-10bb-4783-96b3-6c2681db6964 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:08.569708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42455762-912c-4c27-98f5-feb92b3d16ed admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:08.581674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e321e98-949f-4767-b240-c15a048aa531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:08.583738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e321e98-949f-4767-b240-c15a048aa531 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:08.589183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e321e98-949f-4767-b240-c15a048aa531 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:08.589183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e321e98-949f-4767-b240-c15a048aa531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:08.589183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9e321e98-949f-4767-b240-c15a048aa531 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:08.740078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 878/1768] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:08 2022] POST /identity/v3/users => generated 290 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:08.741398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7d05a-6f2a-4c4b-b29f-178301a633f4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:08.741687 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7d05a-6f2a-4c4b-b29f-178301a633f4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:08.741961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7d05a-6f2a-4c4b-b29f-178301a633f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:08.783427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:08.817038 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a1c7d05a-6f2a-4c4b-b29f-178301a633f4 None None] MFA Rules not processed for user `8d03db170fb6460794e32669456bcdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:08.821430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a1c7d05a-6f2a-4c4b-b29f-178301a633f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:08.914367 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e321e98-949f-4767-b240-c15a048aa531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:08.919216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e321e98-949f-4767-b240-c15a048aa531 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:08.958780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 891/1769] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:47:08.958780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e321e98-949f-4767-b240-c15a048aa531 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:08.987831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e321e98-949f-4767-b240-c15a048aa531 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:09.049807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55793-04a9-47f7-8595-0d907d46be0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:09.060102 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55793-04a9-47f7-8595-0d907d46be0d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:09.064748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55793-04a9-47f7-8595-0d907d46be0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:09.069566 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55793-04a9-47f7-8595-0d907d46be0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:09.069566 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec55793-04a9-47f7-8595-0d907d46be0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:09.095941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec55793-04a9-47f7-8595-0d907d46be0d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:09.113543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 879/1770] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:47:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:09.187426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 892/1771] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:09.191891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7d05a-6f2a-4c4b-b29f-178301a633f4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:09.220242 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7d05a-6f2a-4c4b-b29f-178301a633f4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:09.235933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc4bba-63f0-4d1c-89f5-34f5424a7c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:09.238054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc4bba-63f0-4d1c-89f5-34f5424a7c7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:09.240606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc4bba-63f0-4d1c-89f5-34f5424a7c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:09.245390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc4bba-63f0-4d1c-89f5-34f5424a7c7b admin admin] PATH_INFO: `/v3/projects/202dbcd20e7e47c5a6a5ca7a995d4b57/users/20459c81a2cb4c1d81bdc358660504f9/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:09.287826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc4bba-63f0-4d1c-89f5-34f5424a7c7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=202dbcd20e7e47c5a6a5ca7a995d4b57, user_id=20459c81a2cb4c1d81bdc358660504f9, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:09.298668 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc4bba-63f0-4d1c-89f5-34f5424a7c7b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:09.328918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55793-04a9-47f7-8595-0d907d46be0d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:09.332509 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bec55793-04a9-47f7-8595-0d907d46be0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:09.413209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55793-04a9-47f7-8595-0d907d46be0d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:09.429490 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 880/1772] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:09 2022] PUT /identity/v3/projects/202dbcd20e7e47c5a6a5ca7a995d4b57/users/20459c81a2cb4c1d81bdc358660504f9/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:09.443058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc4bba-63f0-4d1c-89f5-34f5424a7c7b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:09.681896 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:09.683453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:09.683928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:09.684383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] PATH_INFO: `/v3/users/eaaddb757a134fd4803e0b73fb670fce` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:09.686933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaaddb757a134fd4803e0b73fb670fce)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:09.690123 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:09.825915 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc4bba-63f0-4d1c-89f5-34f5424a7c7b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:09.856556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61e3e6-86b1-4e24-94f3-fc8a39f4c8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:09.857205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61e3e6-86b1-4e24-94f3-fc8a39f4c8bd admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:09.857790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61e3e6-86b1-4e24-94f3-fc8a39f4c8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:09.858123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61e3e6-86b1-4e24-94f3-fc8a39f4c8bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:09.858846 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61e3e6-86b1-4e24-94f3-fc8a39f4c8bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:09.860703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61e3e6-86b1-4e24-94f3-fc8a39f4c8bd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:09.888330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 881/1773] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:09.888330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61e3e6-86b1-4e24-94f3-fc8a39f4c8bd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:09.893160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0a61e3e6-86b1-4e24-94f3-fc8a39f4c8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:10.095479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:10.261397 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaaddb757a134fd4803e0b73fb670fce'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:10.262983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaaddb757a134fd4803e0b73fb670fce'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:10.304222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61e3e6-86b1-4e24-94f3-fc8a39f4c8bd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:10.339406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713acc48-b817-42f5-a249-98a3a0ab6fbc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d03db170fb6460794e32669456bcdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:10.357143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713acc48-b817-42f5-a249-98a3a0ab6fbc service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:10.369423 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713acc48-b817-42f5-a249-98a3a0ab6fbc service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:10.378882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaaddb757a134fd4803e0b73fb670fce'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:10.383332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713acc48-b817-42f5-a249-98a3a0ab6fbc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:10.415316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-713acc48-b817-42f5-a249-98a3a0ab6fbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:10.422440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaaddb757a134fd4803e0b73fb670fce'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:10.470223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713acc48-b817-42f5-a249-98a3a0ab6fbc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:10.479917 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713acc48-b817-42f5-a249-98a3a0ab6fbc service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:10.514922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaaddb757a134fd4803e0b73fb670fce'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:10.521086 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 882/1774] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:10.527940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128bad8-9d18-4423-9542-1a36582766e5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:10.530465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128bad8-9d18-4423-9542-1a36582766e5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:10.532650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128bad8-9d18-4423-9542-1a36582766e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:10.538537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 893/1775] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:09 2022] DELETE /identity/v3/users/eaaddb757a134fd4803e0b73fb670fce => generated 0 bytes in 1208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:47:10.540387 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:10.546059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:10.613849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b128bad8-9d18-4423-9542-1a36582766e5 None None] MFA Rules not processed for user `92867e15dcf649adad78bf7a29a996f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:10.629340 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b128bad8-9d18-4423-9542-1a36582766e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:10.783911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:10.900088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 883/1776] 10.222.0.43 () {58 vars in 1015 bytes} [Sun Sep 11 08:47:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:10.904792 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128bad8-9d18-4423-9542-1a36582766e5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:10.910902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b128bad8-9d18-4423-9542-1a36582766e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:10.988493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54822227-5197-4190-8e4f-10364a77e052 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:11.018591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d442d5-d75e-4369-8caa-edf8d97d9090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:11.023118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d442d5-d75e-4369-8caa-edf8d97d9090 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:11.023593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d442d5-d75e-4369-8caa-edf8d97d9090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:11.028816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d442d5-d75e-4369-8caa-edf8d97d9090 admin admin] PATH_INFO: `/v3/projects/202dbcd20e7e47c5a6a5ca7a995d4b57/users/20459c81a2cb4c1d81bdc358660504f9/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:11.045056 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d442d5-d75e-4369-8caa-edf8d97d9090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=202dbcd20e7e47c5a6a5ca7a995d4b57, user_id=20459c81a2cb4c1d81bdc358660504f9, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:11.055348 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d442d5-d75e-4369-8caa-edf8d97d9090 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:11.151342 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 894/1777] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:10 2022] PUT /identity/v3/projects/202dbcd20e7e47c5a6a5ca7a995d4b57/users/20459c81a2cb4c1d81bdc358660504f9/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:11.163709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128bad8-9d18-4423-9542-1a36582766e5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:11.172144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d442d5-d75e-4369-8caa-edf8d97d9090 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:11.174879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-48d442d5-d75e-4369-8caa-edf8d97d9090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:11.480879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d442d5-d75e-4369-8caa-edf8d97d9090 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:11.507653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755f7af-e710-4161-85f9-bb30f65e3b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:11.511684 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755f7af-e710-4161-85f9-bb30f65e3b3c service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:11.513403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755f7af-e710-4161-85f9-bb30f65e3b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:11.513403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755f7af-e710-4161-85f9-bb30f65e3b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:11.535749 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755f7af-e710-4161-85f9-bb30f65e3b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:11.541099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755f7af-e710-4161-85f9-bb30f65e3b3c service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:11.585623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 895/1778] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:11.592265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2481c0-fdfc-417c-a9cd-52e2e146c208 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:11.592589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:11.593324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2481c0-fdfc-417c-a9cd-52e2e146c208 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:11.594187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2481c0-fdfc-417c-a9cd-52e2e146c208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:11.611200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:11.611768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:11.612253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] PATH_INFO: `/v3/users/73a56f8d4421480ea0037459173f709d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:11.637157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a56f8d4421480ea0037459173f709d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:11.651536 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:11.690493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bc2481c0-fdfc-417c-a9cd-52e2e146c208 None None] MFA Rules not processed for user `20459c81a2cb4c1d81bdc358660504f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:11.691892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bc2481c0-fdfc-417c-a9cd-52e2e146c208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:11.939469 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:12.072409 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a56f8d4421480ea0037459173f709d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:12.103462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a56f8d4421480ea0037459173f709d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:12.247488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a56f8d4421480ea0037459173f709d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:12.250575 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a56f8d4421480ea0037459173f709d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:12.270132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 896/1779] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 681 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:12.271677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2481c0-fdfc-417c-a9cd-52e2e146c208 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:12.277870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bc2481c0-fdfc-417c-a9cd-52e2e146c208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:12.278945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a56f8d4421480ea0037459173f709d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:12.308154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 884/1780] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:10 2022] DELETE /identity/v3/users/73a56f8d4421480ea0037459173f709d => generated 0 bytes in 1407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:12.309269 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:12.315907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:12.361739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2481c0-fdfc-417c-a9cd-52e2e146c208 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:12.626022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc82e3-01e7-42df-be49-4c60233eb6a3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:12.637264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3276fef-3533-4de9-b33b-25779f6e0ff0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:12.638371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3276fef-3533-4de9-b33b-25779f6e0ff0 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:12.638866 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3276fef-3533-4de9-b33b-25779f6e0ff0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:12.644549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3276fef-3533-4de9-b33b-25779f6e0ff0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:12.644549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b3276fef-3533-4de9-b33b-25779f6e0ff0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:12.680885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:12.688244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:12.690454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:12.691284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] PATH_INFO: `/v3/projects/c0b804d74cdf44328ab0446a2f485080` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:12.702416 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0b804d74cdf44328ab0446a2f485080)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:12.718323 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:12.862416 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3276fef-3533-4de9-b33b-25779f6e0ff0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:12.865946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3276fef-3533-4de9-b33b-25779f6e0ff0 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:12.904798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 885/1781] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:12.908569 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97521-122c-4ed6-9ec4-63f70cc54b8d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:12.908879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97521-122c-4ed6-9ec4-63f70cc54b8d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:12.909227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97521-122c-4ed6-9ec4-63f70cc54b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:13.030674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0b804d74cdf44328ab0446a2f485080', 'request_id': 'req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:13.035856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f6f97521-122c-4ed6-9ec4-63f70cc54b8d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:13.049018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6f97521-122c-4ed6-9ec4-63f70cc54b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:13.122506 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:13.164230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0b804d74cdf44328ab0446a2f485080', 'request_id': 'req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:13.211128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0b804d74cdf44328ab0446a2f485080', 'request_id': 'req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:13.282212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] The token cache is being invalidate because project c0b804d74cdf44328ab0446a2f485080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:13.282577 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:13.286029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:13.298211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 897/1782] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:12 2022] DELETE /identity/v3/projects/c0b804d74cdf44328ab0446a2f485080 => generated 0 bytes in 1023 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:13.303756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:13.311026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:13.500226 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:13.584324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 886/1783] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 679 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:13.625958 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f97521-122c-4ed6-9ec4-63f70cc54b8d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:13.629253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6f97521-122c-4ed6-9ec4-63f70cc54b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:13.649284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034e1f1-d3e9-4437-a1d1-40edc1930d8e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:13.695030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5df63-9b88-49f8-84ce-80b547001fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:13.696170 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5df63-9b88-49f8-84ce-80b547001fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:13.696611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5df63-9b88-49f8-84ce-80b547001fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:13.698285 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5df63-9b88-49f8-84ce-80b547001fd4 admin admin] PATH_INFO: `/v3/users/571506ba11a7489797ae2c2013c50e68` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:13.703260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5df63-9b88-49f8-84ce-80b547001fd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=571506ba11a7489797ae2c2013c50e68)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:13.707325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5df63-9b88-49f8-84ce-80b547001fd4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:14.077441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f97521-122c-4ed6-9ec4-63f70cc54b8d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:14.124588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024a1a9-9852-4bc3-8d86-013083e62d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:14.138105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024a1a9-9852-4bc3-8d86-013083e62d1c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:14.145474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024a1a9-9852-4bc3-8d86-013083e62d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:14.147224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024a1a9-9852-4bc3-8d86-013083e62d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:14.203054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024a1a9-9852-4bc3-8d86-013083e62d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:14.205558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024a1a9-9852-4bc3-8d86-013083e62d1c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:14.243094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-efa5df63-9b88-49f8-84ce-80b547001fd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '571506ba11a7489797ae2c2013c50e68'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:14.245089 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-efa5df63-9b88-49f8-84ce-80b547001fd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '571506ba11a7489797ae2c2013c50e68'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:14.247442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 887/1784] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:14.351723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-efa5df63-9b88-49f8-84ce-80b547001fd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '571506ba11a7489797ae2c2013c50e68'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:14.379634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-efa5df63-9b88-49f8-84ce-80b547001fd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '571506ba11a7489797ae2c2013c50e68'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:14.476410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-efa5df63-9b88-49f8-84ce-80b547001fd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '571506ba11a7489797ae2c2013c50e68'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:14.509539 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 898/1785] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:13 2022] DELETE /identity/v3/users/571506ba11a7489797ae2c2013c50e68 => generated 0 bytes in 1206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:16.887562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024a1a9-9852-4bc3-8d86-013083e62d1c service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:16.899702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024a1a9-9852-4bc3-8d86-013083e62d1c service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:17.121964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535f824-0cb5-4362-aa3a-c876f8dcaf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:17.122994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f824-0cb5-4362-aa3a-c876f8dcaf56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:17.123413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f824-0cb5-4362-aa3a-c876f8dcaf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:17.124022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f824-0cb5-4362-aa3a-c876f8dcaf56 admin admin] PATH_INFO: `/v3/projects/7d4b0502715d43baa39c77f6d0fa46b3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:17.127781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535f824-0cb5-4362-aa3a-c876f8dcaf56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d4b0502715d43baa39c77f6d0fa46b3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:17.135481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535f824-0cb5-4362-aa3a-c876f8dcaf56 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:17.413644 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1535f824-0cb5-4362-aa3a-c876f8dcaf56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d4b0502715d43baa39c77f6d0fa46b3', 'request_id': 'req-1535f824-0cb5-4362-aa3a-c876f8dcaf56'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:17.530295 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1535f824-0cb5-4362-aa3a-c876f8dcaf56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d4b0502715d43baa39c77f6d0fa46b3', 'request_id': 'req-1535f824-0cb5-4362-aa3a-c876f8dcaf56'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:17.544625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1535f824-0cb5-4362-aa3a-c876f8dcaf56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d4b0502715d43baa39c77f6d0fa46b3', 'request_id': 'req-1535f824-0cb5-4362-aa3a-c876f8dcaf56'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:17.622060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1535f824-0cb5-4362-aa3a-c876f8dcaf56 admin admin] The token cache is being invalidate because project 7d4b0502715d43baa39c77f6d0fa46b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:17.626535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1535f824-0cb5-4362-aa3a-c876f8dcaf56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:17.632639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1535f824-0cb5-4362-aa3a-c876f8dcaf56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:17.640835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 888/1786] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:16 2022] DELETE /identity/v3/projects/7d4b0502715d43baa39c77f6d0fa46b3 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:17.647712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797cf2a-608f-4ec6-a791-568ffaecc679 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:17.648688 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797cf2a-608f-4ec6-a791-568ffaecc679 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:17.649116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797cf2a-608f-4ec6-a791-568ffaecc679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:17.753273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e797cf2a-608f-4ec6-a791-568ffaecc679 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:17.763746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e797cf2a-608f-4ec6-a791-568ffaecc679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:17.789645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:18.033087 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 899/1787] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:18.050499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e7ca9-16aa-4dd3-bde8-b47a7212fb81 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:18.051391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e7ca9-16aa-4dd3-bde8-b47a7212fb81 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:18.052239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e7ca9-16aa-4dd3-bde8-b47a7212fb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:18.134804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e62e7ca9-16aa-4dd3-bde8-b47a7212fb81 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:18.140585 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e62e7ca9-16aa-4dd3-bde8-b47a7212fb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:18.170302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 889/1788] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:18.184636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797cf2a-608f-4ec6-a791-568ffaecc679 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:18.203655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797cf2a-608f-4ec6-a791-568ffaecc679 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:18.220986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb86a1-8dd8-4907-ab3a-3902cdf7ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:18.221668 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb86a1-8dd8-4907-ab3a-3902cdf7ff11 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:18.221946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb86a1-8dd8-4907-ab3a-3902cdf7ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:18.222238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb86a1-8dd8-4907-ab3a-3902cdf7ff11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:18.222971 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb86a1-8dd8-4907-ab3a-3902cdf7ff11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:47:18.223225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb86a1-8dd8-4907-ab3a-3902cdf7ff11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:18.227383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb86a1-8dd8-4907-ab3a-3902cdf7ff11 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:18.271665 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 900/1789] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:47:18.272896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e7ca9-16aa-4dd3-bde8-b47a7212fb81 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:18.289060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e7ca9-16aa-4dd3-bde8-b47a7212fb81 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:18.322913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a8233-4430-44e1-a39c-b0f64c8363a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:18.323654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a8233-4430-44e1-a39c-b0f64c8363a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:18.323934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a8233-4430-44e1-a39c-b0f64c8363a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:18.324206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a8233-4430-44e1-a39c-b0f64c8363a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:18.324912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a8233-4430-44e1-a39c-b0f64c8363a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:18.327035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a8233-4430-44e1-a39c-b0f64c8363a1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:18.445695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb86a1-8dd8-4907-ab3a-3902cdf7ff11 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:18.448695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0ddb86a1-8dd8-4907-ab3a-3902cdf7ff11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:18.492175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb86a1-8dd8-4907-ab3a-3902cdf7ff11 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:18.532179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:18.535377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:18.544868 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 890/1790] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:18 2022] POST /identity/v3/projects => generated 381 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:18.546472 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:18.546761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:18.551025 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a8233-4430-44e1-a39c-b0f64c8363a1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:18.551727 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] PATH_INFO: `/v3/projects/c748ed5480cd417ea9addcc719399b46` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:18.559845 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c748ed5480cd417ea9addcc719399b46)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:18.565327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:18.795601 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a8233-4430-44e1-a39c-b0f64c8363a1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:18.821382 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536ce2c-c639-4da7-b628-7b139b518b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:18.822457 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536ce2c-c639-4da7-b628-7b139b518b51 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:18.823058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536ce2c-c639-4da7-b628-7b139b518b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:18.823979 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536ce2c-c639-4da7-b628-7b139b518b51 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:18.825168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536ce2c-c639-4da7-b628-7b139b518b51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:18.828966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536ce2c-c639-4da7-b628-7b139b518b51 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:18.915803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c748ed5480cd417ea9addcc719399b46', 'request_id': 'req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:19.000628 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 891/1791] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:18 2022] POST /identity/v3/users => generated 291 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:19.004506 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536ce2c-c639-4da7-b628-7b139b518b51 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:19.014856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c748ed5480cd417ea9addcc719399b46', 'request_id': 'req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:19.035934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536ce2c-c639-4da7-b628-7b139b518b51 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:19.046407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c748ed5480cd417ea9addcc719399b46', 'request_id': 'req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:19.098517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] The token cache is being invalidate because project c748ed5480cd417ea9addcc719399b46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:19.099613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:19.101549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:19.113925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 901/1792] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:18 2022] DELETE /identity/v3/projects/c748ed5480cd417ea9addcc719399b46 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:19.118766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:19.122812 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:19.235082 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51846dc6-d0c5-46fa-9666-d0d4927d9e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:19.240400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51846dc6-d0c5-46fa-9666-d0d4927d9e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:19.240821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51846dc6-d0c5-46fa-9666-d0d4927d9e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:19.246795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51846dc6-d0c5-46fa-9666-d0d4927d9e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:19.246795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51846dc6-d0c5-46fa-9666-d0d4927d9e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:19.246795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51846dc6-d0c5-46fa-9666-d0d4927d9e2b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:19.294358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 892/1793] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:19.306123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51846dc6-d0c5-46fa-9666-d0d4927d9e2b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:19.318305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-51846dc6-d0c5-46fa-9666-d0d4927d9e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:19.372922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51846dc6-d0c5-46fa-9666-d0d4927d9e2b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:19.388742 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc52c4-cded-43f9-bf4d-09efb106e397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:19.389405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc52c4-cded-43f9-bf4d-09efb106e397 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:19.389973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc52c4-cded-43f9-bf4d-09efb106e397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:19.390287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc52c4-cded-43f9-bf4d-09efb106e397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:19.390973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc52c4-cded-43f9-bf4d-09efb106e397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:19.393567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc52c4-cded-43f9-bf4d-09efb106e397 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:19.454699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 893/1794] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:19.457623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc52c4-cded-43f9-bf4d-09efb106e397 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:19.494287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc52c4-cded-43f9-bf4d-09efb106e397 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:19.512470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3aa623-dbf9-4d3c-9969-ffdf28d4fb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:19.513258 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3aa623-dbf9-4d3c-9969-ffdf28d4fb6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:19.513445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3aa623-dbf9-4d3c-9969-ffdf28d4fb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:19.513723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3aa623-dbf9-4d3c-9969-ffdf28d4fb6c admin admin] PATH_INFO: `/v3/projects/b1f0f7cba1ac487094c99454bb57e792/users/ddf6d219a5014b98abcb0f2b34341a3b/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:19.556585 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9472e4-ffbf-4b8e-b1dc-a888b04729a5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:19.566270 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:19.567135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:19.567514 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:19.568044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] PATH_INFO: `/v3/users/0c0d740c37414c81a9f045c545eaa54e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:19.569905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0d740c37414c81a9f045c545eaa54e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:19.572931 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:19.582486 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3aa623-dbf9-4d3c-9969-ffdf28d4fb6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f0f7cba1ac487094c99454bb57e792, user_id=ddf6d219a5014b98abcb0f2b34341a3b, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:19.585606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3aa623-dbf9-4d3c-9969-ffdf28d4fb6c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:19.625448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:19.715983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 894/1795] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:19 2022] PUT /identity/v3/projects/b1f0f7cba1ac487094c99454bb57e792/users/ddf6d219a5014b98abcb0f2b34341a3b/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:19.739680 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab4f2d-37e4-4151-9a06-76335095fd5e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:19.739975 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab4f2d-37e4-4151-9a06-76335095fd5e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:19.740256 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab4f2d-37e4-4151-9a06-76335095fd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:19.818490 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-eaab4f2d-37e4-4151-9a06-76335095fd5e None None] MFA Rules not processed for user `ddf6d219a5014b98abcb0f2b34341a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:19.820083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eaab4f2d-37e4-4151-9a06-76335095fd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:19.936590 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:19.986390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0d740c37414c81a9f045c545eaa54e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:19.988322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0d740c37414c81a9f045c545eaa54e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:20.060415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 895/1796] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:20.067229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab4f2d-37e4-4151-9a06-76335095fd5e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:20.091535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0d740c37414c81a9f045c545eaa54e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:20.104961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0d740c37414c81a9f045c545eaa54e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:20.152654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0d740c37414c81a9f045c545eaa54e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:20.176943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 902/1797] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:19 2022] DELETE /identity/v3/users/0c0d740c37414c81a9f045c545eaa54e => generated 0 bytes in 1060 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:20.247532 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab4f2d-37e4-4151-9a06-76335095fd5e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:20.434063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0fe23-b10d-4a1b-99f3-8e904d816c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:20.434781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0fe23-b10d-4a1b-99f3-8e904d816c9c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:20.435062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0fe23-b10d-4a1b-99f3-8e904d816c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:20.435333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0fe23-b10d-4a1b-99f3-8e904d816c9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:20.437753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0fe23-b10d-4a1b-99f3-8e904d816c9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:20.446056 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0fe23-b10d-4a1b-99f3-8e904d816c9c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:20.702750 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 896/1798] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:20 2022] POST /identity/v3/projects => generated 383 bytes in 636 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:20.708406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:20.710248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:20.759502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5ead8-35ee-4abe-92d4-5eb77fd21ed2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:21.034671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bc00b-062e-4891-966b-77e523e3b1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:21.040430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bc00b-062e-4891-966b-77e523e3b1c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:21.041017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bc00b-062e-4891-966b-77e523e3b1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:21.041508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bc00b-062e-4891-966b-77e523e3b1c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:21.042554 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bc00b-062e-4891-966b-77e523e3b1c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:21.047921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bc00b-062e-4891-966b-77e523e3b1c2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:21.316158 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 903/1799] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:20 2022] POST /identity/v3/users => generated 298 bytes in 608 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:21.317527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0fe23-b10d-4a1b-99f3-8e904d816c9c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:21.352893 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bc00b-062e-4891-966b-77e523e3b1c2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:21.359648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f48bc00b-062e-4891-966b-77e523e3b1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:21.365978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0fe23-b10d-4a1b-99f3-8e904d816c9c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:21.401712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2aa83-0ed5-4b5f-b382-fe7701eb2532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:21.402467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2aa83-0ed5-4b5f-b382-fe7701eb2532 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:21.413449 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2aa83-0ed5-4b5f-b382-fe7701eb2532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:21.414446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2aa83-0ed5-4b5f-b382-fe7701eb2532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:21.419614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2aa83-0ed5-4b5f-b382-fe7701eb2532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:21.424997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bc00b-062e-4891-966b-77e523e3b1c2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:21.428895 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2aa83-0ed5-4b5f-b382-fe7701eb2532 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:21.458620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b88e89-4f3c-406c-87f6-6a0b14f8ada8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:21.463716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b88e89-4f3c-406c-87f6-6a0b14f8ada8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:21.464186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b88e89-4f3c-406c-87f6-6a0b14f8ada8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:21.464621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b88e89-4f3c-406c-87f6-6a0b14f8ada8 admin admin] PATH_INFO: `/v3/users/9e9de6cc5a6c4e46801665eb7109fc77` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:21.467843 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b88e89-4f3c-406c-87f6-6a0b14f8ada8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9de6cc5a6c4e46801665eb7109fc77)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:21.470121 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b88e89-4f3c-406c-87f6-6a0b14f8ada8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:21.498610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 897/1800] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:21.500669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2aa83-0ed5-4b5f-b382-fe7701eb2532 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:21.559819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2aa83-0ed5-4b5f-b382-fe7701eb2532 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:21.587423 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cebff-16ff-4aae-ad8c-7e73097a0de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:21.588533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cebff-16ff-4aae-ad8c-7e73097a0de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:21.588956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cebff-16ff-4aae-ad8c-7e73097a0de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:21.589397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cebff-16ff-4aae-ad8c-7e73097a0de6 admin admin] PATH_INFO: `/v3/projects/d2ad4b447fc14e9a993abc9b937439b6/users/174e02b658d94a4fab561fb4a4a8fa5e/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:21.668376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cebff-16ff-4aae-ad8c-7e73097a0de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ad4b447fc14e9a993abc9b937439b6, user_id=174e02b658d94a4fab561fb4a4a8fa5e, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:21.684036 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cebff-16ff-4aae-ad8c-7e73097a0de6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:21.893996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 898/1801] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:21 2022] PUT /identity/v3/projects/d2ad4b447fc14e9a993abc9b937439b6/users/174e02b658d94a4fab561fb4a4a8fa5e/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:21.896388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab191e8-235f-4f3a-ab0f-171bd553cd1b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:21.896963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab191e8-235f-4f3a-ab0f-171bd553cd1b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:21.897481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab191e8-235f-4f3a-ab0f-171bd553cd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:21.897997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-95b88e89-4f3c-406c-87f6-6a0b14f8ada8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9de6cc5a6c4e46801665eb7109fc77'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:21.899569 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-95b88e89-4f3c-406c-87f6-6a0b14f8ada8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9de6cc5a6c4e46801665eb7109fc77'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:21.929957 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3ab191e8-235f-4f3a-ab0f-171bd553cd1b None None] MFA Rules not processed for user `3abba619decf448db477533c663004bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:21.931774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3ab191e8-235f-4f3a-ab0f-171bd553cd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:22.044772 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:22.100489 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-95b88e89-4f3c-406c-87f6-6a0b14f8ada8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9de6cc5a6c4e46801665eb7109fc77'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:22.116587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-95b88e89-4f3c-406c-87f6-6a0b14f8ada8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9de6cc5a6c4e46801665eb7109fc77'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:22.141620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 899/1802] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:22.143415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a828f-616d-4758-807b-d839136e2376 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:22.147637 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a828f-616d-4758-807b-d839136e2376 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:22.148173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a828f-616d-4758-807b-d839136e2376 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:22.152942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 900/1803] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:47:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:47:22.154075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a828f-616d-4758-807b-d839136e2376 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:22.158180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ab9a828f-616d-4758-807b-d839136e2376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:22.207350 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-95b88e89-4f3c-406c-87f6-6a0b14f8ada8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9de6cc5a6c4e46801665eb7109fc77'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:22.219389 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=17,keystone:SELECT=78,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:22.242773 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 904/1804] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:21 2022] DELETE /identity/v3/users/9e9de6cc5a6c4e46801665eb7109fc77 => generated 0 bytes in 887 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:22.244940 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38fdfd-eacf-406c-a817-f9bbaea2b1ea None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:22.245225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38fdfd-eacf-406c-a817-f9bbaea2b1ea None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:22.245644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38fdfd-eacf-406c-a817-f9bbaea2b1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:22.300695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8d38fdfd-eacf-406c-a817-f9bbaea2b1ea None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:22.302767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8d38fdfd-eacf-406c-a817-f9bbaea2b1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:22.370458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a828f-616d-4758-807b-d839136e2376 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:22.409214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 905/1805] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:22.451993 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38fdfd-eacf-406c-a817-f9bbaea2b1ea None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:22.454127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8d38fdfd-eacf-406c-a817-f9bbaea2b1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:22.532670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d53c61-c850-43b3-9878-ba8932a36dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:22.533445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d53c61-c850-43b3-9878-ba8932a36dcd admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:22.534041 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d53c61-c850-43b3-9878-ba8932a36dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:22.534526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d53c61-c850-43b3-9878-ba8932a36dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:22.535359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d53c61-c850-43b3-9878-ba8932a36dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:22.536994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d53c61-c850-43b3-9878-ba8932a36dcd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:22.567929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 901/1806] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:22.571832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50c6f6-4948-424d-ae4e-4c98473a3252 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:22.571832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50c6f6-4948-424d-ae4e-4c98473a3252 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:22.571832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50c6f6-4948-424d-ae4e-4c98473a3252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:22.612734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bb50c6f6-4948-424d-ae4e-4c98473a3252 None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:22.614000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bb50c6f6-4948-424d-ae4e-4c98473a3252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:22.626309 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38fdfd-eacf-406c-a817-f9bbaea2b1ea None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:22.634086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c9c20-8464-42f6-9efa-c0388210ecaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:22.634794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c9c20-8464-42f6-9efa-c0388210ecaf service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:22.635068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c9c20-8464-42f6-9efa-c0388210ecaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:22.635335 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c9c20-8464-42f6-9efa-c0388210ecaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:22.658884 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c9c20-8464-42f6-9efa-c0388210ecaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:22.661014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c9c20-8464-42f6-9efa-c0388210ecaf service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:22.689211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 906/1807] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:47:22.693241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c9c20-8464-42f6-9efa-c0388210ecaf service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:22.725636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c9c20-8464-42f6-9efa-c0388210ecaf service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:22.742411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0e437-e8ea-4453-9820-e9409f78436e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:22.749010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0e437-e8ea-4453-9820-e9409f78436e admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:22.749010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0e437-e8ea-4453-9820-e9409f78436e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:22.749010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0e437-e8ea-4453-9820-e9409f78436e admin admin] PATH_INFO: `/v3/projects/d2ad4b447fc14e9a993abc9b937439b6/users/174e02b658d94a4fab561fb4a4a8fa5e/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:22.753146 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0e437-e8ea-4453-9820-e9409f78436e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ad4b447fc14e9a993abc9b937439b6, user_id=174e02b658d94a4fab561fb4a4a8fa5e, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:22.760471 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0e437-e8ea-4453-9820-e9409f78436e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:22.797521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 907/1808] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:22 2022] PUT /identity/v3/projects/d2ad4b447fc14e9a993abc9b937439b6/users/174e02b658d94a4fab561fb4a4a8fa5e/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:22.800067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0e437-e8ea-4453-9820-e9409f78436e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:22.804864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f5c0e437-e8ea-4453-9820-e9409f78436e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:22.850834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 902/1809] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:47:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:22.853095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19ba76-28eb-41ae-b55c-87045829d192 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:22.853515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19ba76-28eb-41ae-b55c-87045829d192 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:22.853855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19ba76-28eb-41ae-b55c-87045829d192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:22.913293 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ce19ba76-28eb-41ae-b55c-87045829d192 None None] MFA Rules not processed for user `174e02b658d94a4fab561fb4a4a8fa5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:22.915009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ce19ba76-28eb-41ae-b55c-87045829d192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:23.063712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:23.077912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0e437-e8ea-4453-9820-e9409f78436e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:23.095589 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3b835-cf12-46de-9d21-07dc8623efbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:23.099519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3b835-cf12-46de-9d21-07dc8623efbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:23.101055 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3b835-cf12-46de-9d21-07dc8623efbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:23.102366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3b835-cf12-46de-9d21-07dc8623efbb admin admin] PATH_INFO: `/v3/projects/321a2870a48d4309836fd2e5c88169f0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:23.116564 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3b835-cf12-46de-9d21-07dc8623efbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=321a2870a48d4309836fd2e5c88169f0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:23.119951 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3b835-cf12-46de-9d21-07dc8623efbb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:23.365710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 903/1810] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:22 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 514 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:23.382757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70c3b835-cf12-46de-9d21-07dc8623efbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '321a2870a48d4309836fd2e5c88169f0', 'request_id': 'req-70c3b835-cf12-46de-9d21-07dc8623efbb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:23.388741 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19ba76-28eb-41ae-b55c-87045829d192 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:23.391291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ce19ba76-28eb-41ae-b55c-87045829d192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:23.506413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70c3b835-cf12-46de-9d21-07dc8623efbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '321a2870a48d4309836fd2e5c88169f0', 'request_id': 'req-70c3b835-cf12-46de-9d21-07dc8623efbb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:23.550177 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70c3b835-cf12-46de-9d21-07dc8623efbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '321a2870a48d4309836fd2e5c88169f0', 'request_id': 'req-70c3b835-cf12-46de-9d21-07dc8623efbb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:23.584412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70c3b835-cf12-46de-9d21-07dc8623efbb admin admin] The token cache is being invalidate because project 321a2870a48d4309836fd2e5c88169f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:23.584856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70c3b835-cf12-46de-9d21-07dc8623efbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:23.585702 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70c3b835-cf12-46de-9d21-07dc8623efbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:23.588791 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 908/1811] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:22 2022] DELETE /identity/v3/projects/321a2870a48d4309836fd2e5c88169f0 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:23.596191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1066da-ceef-49cb-906d-4a216dbe17a0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:23.596608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1066da-ceef-49cb-906d-4a216dbe17a0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:23.597167 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1066da-ceef-49cb-906d-4a216dbe17a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:23.643664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9d1066da-ceef-49cb-906d-4a216dbe17a0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:23.648073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9d1066da-ceef-49cb-906d-4a216dbe17a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:23.671053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19ba76-28eb-41ae-b55c-87045829d192 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:23.849718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 909/1812] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:23.853983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1066da-ceef-49cb-906d-4a216dbe17a0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:23.853983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9d1066da-ceef-49cb-906d-4a216dbe17a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:23.911887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:23.957614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af0a19-ea0e-416c-becf-ba64f3e9d692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:23.958885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af0a19-ea0e-416c-becf-ba64f3e9d692 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:23.959410 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af0a19-ea0e-416c-becf-ba64f3e9d692 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:23.959982 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af0a19-ea0e-416c-becf-ba64f3e9d692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:23.964360 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-56af0a19-ea0e-416c-becf-ba64f3e9d692 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:24.229561 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af0a19-ea0e-416c-becf-ba64f3e9d692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:24.231408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af0a19-ea0e-416c-becf-ba64f3e9d692 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:24.269694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 904/1813] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 882 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:47:24.272350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de5abb-87e9-42e9-a85b-33b72929f4d1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:24.272718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de5abb-87e9-42e9-a85b-33b72929f4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:24.273155 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de5abb-87e9-42e9-a85b-33b72929f4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:24.296422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:24.325535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-09de5abb-87e9-42e9-a85b-33b72929f4d1 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:24.329478 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-09de5abb-87e9-42e9-a85b-33b72929f4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:24.333813 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1066da-ceef-49cb-906d-4a216dbe17a0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:24.351549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa52036-0efc-4853-acae-71109555efa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:24.352211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa52036-0efc-4853-acae-71109555efa2 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:24.352496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa52036-0efc-4853-acae-71109555efa2 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:24.353009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa52036-0efc-4853-acae-71109555efa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:24.356564 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aaa52036-0efc-4853-acae-71109555efa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:24.376690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa52036-0efc-4853-acae-71109555efa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:24.383315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa52036-0efc-4853-acae-71109555efa2 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:24.387853 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 905/1814] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:24.394342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dce1b-f8ff-4721-abeb-44993ff909e0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:24.395426 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dce1b-f8ff-4721-abeb-44993ff909e0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:24.395426 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dce1b-f8ff-4721-abeb-44993ff909e0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:24.400934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 910/1816] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:24.401271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 906/1816] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:47:24 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:47:24.402167 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa52036-0efc-4853-acae-71109555efa2 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:24.422757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa52036-0efc-4853-acae-71109555efa2 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:24.430895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df83f18-709e-47eb-8670-453b52d5d67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:24.432956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df83f18-709e-47eb-8670-453b52d5d67b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:24.433435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df83f18-709e-47eb-8670-453b52d5d67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:24.433951 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df83f18-709e-47eb-8670-453b52d5d67b admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:24.435106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df83f18-709e-47eb-8670-453b52d5d67b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:47:24.435862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df83f18-709e-47eb-8670-453b52d5d67b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:24.438982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df83f18-709e-47eb-8670-453b52d5d67b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:24.466721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 911/1817] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:47:24.471346 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6dce1b-f8ff-4721-abeb-44993ff909e0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:24.474069 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469feb5e-96a6-42be-b5b4-af7de83792a2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:24.474389 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469feb5e-96a6-42be-b5b4-af7de83792a2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:24.474668 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469feb5e-96a6-42be-b5b4-af7de83792a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:24.506963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6dce1b-f8ff-4721-abeb-44993ff909e0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:24.544054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59118c-263c-4032-b291-8087c58c1a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:24.544054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59118c-263c-4032-b291-8087c58c1a0d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:24.544054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59118c-263c-4032-b291-8087c58c1a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:24.544054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59118c-263c-4032-b291-8087c58c1a0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:24.544931 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59118c-263c-4032-b291-8087c58c1a0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:24.546369 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59118c-263c-4032-b291-8087c58c1a0d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:24.566238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-469feb5e-96a6-42be-b5b4-af7de83792a2 None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:24.568955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-469feb5e-96a6-42be-b5b4-af7de83792a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:24.613523 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 912/1818] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:47:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:24.614453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469feb5e-96a6-42be-b5b4-af7de83792a2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:24.621397 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-469feb5e-96a6-42be-b5b4-af7de83792a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:24.649511 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469feb5e-96a6-42be-b5b4-af7de83792a2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:24.674648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:24.676556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:24.676915 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:24.677425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] PATH_INFO: `/v3/projects/45c18c9452394e9fbdfc1dc38e714f86` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:24.692731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c18c9452394e9fbdfc1dc38e714f86)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:24.704620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:24.748215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 907/1819] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:24 2022] POST /identity/v3/projects => generated 383 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:24.749851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59118c-263c-4032-b291-8087c58c1a0d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:24.754861 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ce59118c-263c-4032-b291-8087c58c1a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:24.785418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:24.970891 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c18c9452394e9fbdfc1dc38e714f86', 'request_id': 'req-682a76c5-e119-49bc-9986-f322844e81d6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:24.988731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59118c-263c-4032-b291-8087c58c1a0d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:25.224541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c18c9452394e9fbdfc1dc38e714f86', 'request_id': 'req-682a76c5-e119-49bc-9986-f322844e81d6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:25.237671 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec9fad-42e1-4ab1-bbc3-29e99f44e72e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:25.238767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec9fad-42e1-4ab1-bbc3-29e99f44e72e service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:25.239447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec9fad-42e1-4ab1-bbc3-29e99f44e72e service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:25.239797 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec9fad-42e1-4ab1-bbc3-29e99f44e72e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:25.247590 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fbec9fad-42e1-4ab1-bbc3-29e99f44e72e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:25.266323 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c18c9452394e9fbdfc1dc38e714f86', 'request_id': 'req-682a76c5-e119-49bc-9986-f322844e81d6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:25.309989 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] The token cache is being invalidate because project 45c18c9452394e9fbdfc1dc38e714f86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:25.315592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:25.316426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:25.320726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 913/1820] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:24 2022] DELETE /identity/v3/projects/45c18c9452394e9fbdfc1dc38e714f86 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:25.321491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:25.323628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:25.547713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec9fad-42e1-4ab1-bbc3-29e99f44e72e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:25.557978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec9fad-42e1-4ab1-bbc3-29e99f44e72e service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:25.584261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a76c5-e119-49bc-9986-f322844e81d6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:25.599592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694429e-dd21-46e1-9775-cc05dbd3906b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:25.601887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694429e-dd21-46e1-9775-cc05dbd3906b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:25.602603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694429e-dd21-46e1-9775-cc05dbd3906b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:25.603215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694429e-dd21-46e1-9775-cc05dbd3906b admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:25.604953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694429e-dd21-46e1-9775-cc05dbd3906b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:25.613206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 908/1821] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 864 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:25.616018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec9fad-42e1-4ab1-bbc3-29e99f44e72e service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:25.618740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694429e-dd21-46e1-9775-cc05dbd3906b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:25.664887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec9fad-42e1-4ab1-bbc3-29e99f44e72e service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:25.685113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:25.686391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:25.686906 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:25.687355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] PATH_INFO: `/v3/users/20459c81a2cb4c1d81bdc358660504f9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:25.694691 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20459c81a2cb4c1d81bdc358660504f9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:25.699149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:25.818382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 914/1822] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:25 2022] POST /identity/v3/users => generated 292 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:25.826257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694429e-dd21-46e1-9775-cc05dbd3906b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:25.873565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694429e-dd21-46e1-9775-cc05dbd3906b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:25.916216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755592b-406d-46ef-9ddd-07d4d502cabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:25.919199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755592b-406d-46ef-9ddd-07d4d502cabd admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:25.919703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755592b-406d-46ef-9ddd-07d4d502cabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:25.920201 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755592b-406d-46ef-9ddd-07d4d502cabd admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:25.927214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755592b-406d-46ef-9ddd-07d4d502cabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:25.929982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755592b-406d-46ef-9ddd-07d4d502cabd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:25.963575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 915/1823] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:25.965851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755592b-406d-46ef-9ddd-07d4d502cabd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:26.012649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755592b-406d-46ef-9ddd-07d4d502cabd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:26.021273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5de661-0526-4713-b183-05803e21e9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:26.022020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5de661-0526-4713-b183-05803e21e9f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:26.023613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5de661-0526-4713-b183-05803e21e9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:26.024014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5de661-0526-4713-b183-05803e21e9f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:26.030562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5de661-0526-4713-b183-05803e21e9f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:26.030562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5de661-0526-4713-b183-05803e21e9f5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:26.053002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 916/1824] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:26.055594 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5de661-0526-4713-b183-05803e21e9f5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:26.070086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5de661-0526-4713-b183-05803e21e9f5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:26.078306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f7098-f77a-471a-8e5d-30d8d1a7996f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:26.082244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f7098-f77a-471a-8e5d-30d8d1a7996f admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:26.082244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f7098-f77a-471a-8e5d-30d8d1a7996f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:26.082244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f7098-f77a-471a-8e5d-30d8d1a7996f admin admin] PATH_INFO: `/v3/projects/cbd52b8041114a18bb07601530a9a5a2/users/48afe85e198748b1ac45e4f86dad33ef/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:26.114352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f7098-f77a-471a-8e5d-30d8d1a7996f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd52b8041114a18bb07601530a9a5a2, user_id=48afe85e198748b1ac45e4f86dad33ef, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:26.118944 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f7098-f77a-471a-8e5d-30d8d1a7996f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:26.130670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:26.133767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20459c81a2cb4c1d81bdc358660504f9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:26.165447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20459c81a2cb4c1d81bdc358660504f9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:26.292176 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20459c81a2cb4c1d81bdc358660504f9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:26.310293 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20459c81a2cb4c1d81bdc358660504f9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:26.310597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 917/1825] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:26 2022] PUT /identity/v3/projects/cbd52b8041114a18bb07601530a9a5a2/users/48afe85e198748b1ac45e4f86dad33ef/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:47:26.318661 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20459c81a2cb4c1d81bdc358660504f9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:26.320958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d90060-6bb8-4c4a-993a-39e9173c387a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:26.321315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d90060-6bb8-4c4a-993a-39e9173c387a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:26.321590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d90060-6bb8-4c4a-993a-39e9173c387a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:26.372758 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 909/1826] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:25 2022] DELETE /identity/v3/users/20459c81a2cb4c1d81bdc358660504f9 => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:26.443566 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-33d90060-6bb8-4c4a-993a-39e9173c387a None None] MFA Rules not processed for user `48afe85e198748b1ac45e4f86dad33ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:26.445008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-33d90060-6bb8-4c4a-993a-39e9173c387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:26.539838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:26.574879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 918/1827] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:26.581513 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:26.583077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:26.792633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d90060-6bb8-4c4a-993a-39e9173c387a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:26.795418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-33d90060-6bb8-4c4a-993a-39e9173c387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:26.850186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca02be94-c22e-4e26-8121-69f12950ab93 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:26.875680 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62922c1-8779-4314-812c-899b8bfb13ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:26.879296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62922c1-8779-4314-812c-899b8bfb13ed admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:26.879661 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62922c1-8779-4314-812c-899b8bfb13ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:26.879935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62922c1-8779-4314-812c-899b8bfb13ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:26.880832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62922c1-8779-4314-812c-899b8bfb13ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:26.887408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62922c1-8779-4314-812c-899b8bfb13ed admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:26.927227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d90060-6bb8-4c4a-993a-39e9173c387a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:26.938956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc833de-d0fa-45ca-b5df-55a64c4a4362 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:26.941138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc833de-d0fa-45ca-b5df-55a64c4a4362 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:26.942957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc833de-d0fa-45ca-b5df-55a64c4a4362 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:26.944073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc833de-d0fa-45ca-b5df-55a64c4a4362 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:26.949475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-efc833de-d0fa-45ca-b5df-55a64c4a4362 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:26.974629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc833de-d0fa-45ca-b5df-55a64c4a4362 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:26.977894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc833de-d0fa-45ca-b5df-55a64c4a4362 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:27.000818 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 919/1828] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:47:27.366103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 910/1829] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:26 2022] POST /identity/v3/projects => generated 383 bytes in 785 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:47:27.373257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc833de-d0fa-45ca-b5df-55a64c4a4362 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:27.420635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc833de-d0fa-45ca-b5df-55a64c4a4362 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:27.690613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ea658-9f48-458c-acab-00e692ef66c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:27.703377 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ea658-9f48-458c-acab-00e692ef66c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:27.703842 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ea658-9f48-458c-acab-00e692ef66c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:27.704345 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ea658-9f48-458c-acab-00e692ef66c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:27.711174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ea658-9f48-458c-acab-00e692ef66c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:27.740228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ea658-9f48-458c-acab-00e692ef66c1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:27.916585 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 920/1830] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:27 2022] POST /identity/v3/users => generated 298 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:27.918625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62922c1-8779-4314-812c-899b8bfb13ed admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:27.937790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62922c1-8779-4314-812c-899b8bfb13ed admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:27.948728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f820c9-9427-4ade-9b84-285ab3b13f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:27.949884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f820c9-9427-4ade-9b84-285ab3b13f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:27.952934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f820c9-9427-4ade-9b84-285ab3b13f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:27.952934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f820c9-9427-4ade-9b84-285ab3b13f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:27.952934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f820c9-9427-4ade-9b84-285ab3b13f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:27.955309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f820c9-9427-4ade-9b84-285ab3b13f1c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:27.979446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 911/1831] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:27.991398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ea658-9f48-458c-acab-00e692ef66c1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:28.012754 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ea658-9f48-458c-acab-00e692ef66c1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:28.018815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999b527-1dff-4bad-b15c-0d1c6739d7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:28.019581 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999b527-1dff-4bad-b15c-0d1c6739d7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:28.020146 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999b527-1dff-4bad-b15c-0d1c6739d7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:28.020431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999b527-1dff-4bad-b15c-0d1c6739d7eb admin admin] PATH_INFO: `/v3/projects/35541df7f55a467db29ece8f8b01c1e2/users/5136406fb78a477089e6538115cb58e9/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:28.040494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c999b527-1dff-4bad-b15c-0d1c6739d7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=35541df7f55a467db29ece8f8b01c1e2, user_id=5136406fb78a477089e6538115cb58e9, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:28.044333 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c999b527-1dff-4bad-b15c-0d1c6739d7eb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:28.180874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 921/1832] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:27 2022] PUT /identity/v3/projects/35541df7f55a467db29ece8f8b01c1e2/users/5136406fb78a477089e6538115cb58e9/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:47:28.185556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f820c9-9427-4ade-9b84-285ab3b13f1c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:28.207035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f820c9-9427-4ade-9b84-285ab3b13f1c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:28.321450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95112cf2-d83d-4578-ad7b-75120dcc5213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:28.327160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95112cf2-d83d-4578-ad7b-75120dcc5213 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:28.327160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95112cf2-d83d-4578-ad7b-75120dcc5213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:28.327160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95112cf2-d83d-4578-ad7b-75120dcc5213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:28.327160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95112cf2-d83d-4578-ad7b-75120dcc5213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:28.327160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95112cf2-d83d-4578-ad7b-75120dcc5213 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:28.338181 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 912/1833] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:47:28.343372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999b527-1dff-4bad-b15c-0d1c6739d7eb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:28.355204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999b527-1dff-4bad-b15c-0d1c6739d7eb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:28.361097 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8affa-72a3-4541-a918-3b5c051e484e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:28.361961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8affa-72a3-4541-a918-3b5c051e484e admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:28.362314 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8affa-72a3-4541-a918-3b5c051e484e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:28.362575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8affa-72a3-4541-a918-3b5c051e484e admin admin] PATH_INFO: `/v3/projects/35541df7f55a467db29ece8f8b01c1e2/users/5136406fb78a477089e6538115cb58e9/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:28.364762 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8affa-72a3-4541-a918-3b5c051e484e admin admin] RBAC: Authorizing `identity:create_grant(project_id=35541df7f55a467db29ece8f8b01c1e2, user_id=5136406fb78a477089e6538115cb58e9, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:28.368096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8affa-72a3-4541-a918-3b5c051e484e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:28.381952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 922/1834] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:28 2022] PUT /identity/v3/projects/35541df7f55a467db29ece8f8b01c1e2/users/5136406fb78a477089e6538115cb58e9/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:47:28.401791 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8affa-72a3-4541-a918-3b5c051e484e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:28.402174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272eaa0-a2a6-41eb-91eb-a2d75276523f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:28.403084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272eaa0-a2a6-41eb-91eb-a2d75276523f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:28.403969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f272eaa0-a2a6-41eb-91eb-a2d75276523f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:28.404950 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a1c8affa-72a3-4541-a918-3b5c051e484e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:28.456807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f272eaa0-a2a6-41eb-91eb-a2d75276523f None None] MFA Rules not processed for user `5136406fb78a477089e6538115cb58e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:28.458605 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f272eaa0-a2a6-41eb-91eb-a2d75276523f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:28.521260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8affa-72a3-4541-a918-3b5c051e484e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:28.540136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f164730-9a03-46ba-8e65-554d72a90c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:28.543024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f164730-9a03-46ba-8e65-554d72a90c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:28.545706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f164730-9a03-46ba-8e65-554d72a90c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:28.546713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f164730-9a03-46ba-8e65-554d72a90c78 admin admin] PATH_INFO: `/v3/projects/202dbcd20e7e47c5a6a5ca7a995d4b57` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:28.568138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f164730-9a03-46ba-8e65-554d72a90c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=202dbcd20e7e47c5a6a5ca7a995d4b57)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:28.568138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f164730-9a03-46ba-8e65-554d72a90c78 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:28.611089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:28.631474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 913/1835] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:28.651823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272eaa0-a2a6-41eb-91eb-a2d75276523f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:28.654008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f272eaa0-a2a6-41eb-91eb-a2d75276523f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:28.731120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f272eaa0-a2a6-41eb-91eb-a2d75276523f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:28.735733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd6f50-eb4d-4645-94db-e6c25faada32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:28.736628 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd6f50-eb4d-4645-94db-e6c25faada32 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:28.736992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd6f50-eb4d-4645-94db-e6c25faada32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:28.737349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd6f50-eb4d-4645-94db-e6c25faada32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:28.825813 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6f164730-9a03-46ba-8e65-554d72a90c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '202dbcd20e7e47c5a6a5ca7a995d4b57', 'request_id': 'req-6f164730-9a03-46ba-8e65-554d72a90c78'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:28.884220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd6f50-eb4d-4645-94db-e6c25faada32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:28.887756 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd6f50-eb4d-4645-94db-e6c25faada32 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:28.903367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 914/1836] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:47:28.957390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6f164730-9a03-46ba-8e65-554d72a90c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '202dbcd20e7e47c5a6a5ca7a995d4b57', 'request_id': 'req-6f164730-9a03-46ba-8e65-554d72a90c78'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:28.968745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6f164730-9a03-46ba-8e65-554d72a90c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '202dbcd20e7e47c5a6a5ca7a995d4b57', 'request_id': 'req-6f164730-9a03-46ba-8e65-554d72a90c78'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:29.028264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6f164730-9a03-46ba-8e65-554d72a90c78 admin admin] The token cache is being invalidate because project 202dbcd20e7e47c5a6a5ca7a995d4b57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:29.028264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6f164730-9a03-46ba-8e65-554d72a90c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:29.028264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6f164730-9a03-46ba-8e65-554d72a90c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:29.028264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 923/1837] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:28 2022] DELETE /identity/v3/projects/202dbcd20e7e47c5a6a5ca7a995d4b57 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:47:29.029739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eadb4b-2856-4444-b49f-b218f991e1e6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:29.029739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eadb4b-2856-4444-b49f-b218f991e1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:29.029739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eadb4b-2856-4444-b49f-b218f991e1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:29.075436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b3eadb4b-2856-4444-b49f-b218f991e1e6 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:29.077289 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b3eadb4b-2856-4444-b49f-b218f991e1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:29.122940 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:29.227088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 915/1838] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:47:29.245018 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22d413-baff-48b8-8b2a-14844625d90f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:29.245305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22d413-baff-48b8-8b2a-14844625d90f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:29.245584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22d413-baff-48b8-8b2a-14844625d90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:29.276979 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2e22d413-baff-48b8-8b2a-14844625d90f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:29.278283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2e22d413-baff-48b8-8b2a-14844625d90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:29.292352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 924/1839] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:29.294128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eadb4b-2856-4444-b49f-b218f991e1e6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:29.315964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eadb4b-2856-4444-b49f-b218f991e1e6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:29.330847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84f2f5-ffdb-4935-b25d-46aefa302246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:29.331574 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84f2f5-ffdb-4935-b25d-46aefa302246 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:29.331760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84f2f5-ffdb-4935-b25d-46aefa302246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:29.332007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84f2f5-ffdb-4935-b25d-46aefa302246 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:29.332672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84f2f5-ffdb-4935-b25d-46aefa302246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:47:29.333200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84f2f5-ffdb-4935-b25d-46aefa302246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:29.334881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84f2f5-ffdb-4935-b25d-46aefa302246 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:29.353669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 916/1840] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:47:29.367564 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22d413-baff-48b8-8b2a-14844625d90f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:29.376784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22d413-baff-48b8-8b2a-14844625d90f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:29.383302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7826116-c7f4-4137-bd1d-f428dd87be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:29.385149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7826116-c7f4-4137-bd1d-f428dd87be3b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:29.385817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7826116-c7f4-4137-bd1d-f428dd87be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:29.386440 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7826116-c7f4-4137-bd1d-f428dd87be3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:29.387787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7826116-c7f4-4137-bd1d-f428dd87be3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:29.391993 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7826116-c7f4-4137-bd1d-f428dd87be3b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:29.553085 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 925/1841] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:29 2022] POST /identity/v3/projects => generated 367 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:47:29.562883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84f2f5-ffdb-4935-b25d-46aefa302246 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:29.588936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84f2f5-ffdb-4935-b25d-46aefa302246 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:29.726005 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45876c-b263-4903-9b71-b12751a3a77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:29.727544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45876c-b263-4903-9b71-b12751a3a77d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:29.728193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45876c-b263-4903-9b71-b12751a3a77d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:29.728986 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45876c-b263-4903-9b71-b12751a3a77d admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:29.730889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45876c-b263-4903-9b71-b12751a3a77d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:29.736066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45876c-b263-4903-9b71-b12751a3a77d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:29.939068 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 917/1842] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:29 2022] POST /identity/v3/users => generated 284 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:47:29.941167 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7826116-c7f4-4137-bd1d-f428dd87be3b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:29.965777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7826116-c7f4-4137-bd1d-f428dd87be3b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:29.985482 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27a623-0669-4c7b-af56-0a0183fe9ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:29.985482 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27a623-0669-4c7b-af56-0a0183fe9ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:29.985482 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27a623-0669-4c7b-af56-0a0183fe9ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:29.985482 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27a623-0669-4c7b-af56-0a0183fe9ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:29.987227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27a623-0669-4c7b-af56-0a0183fe9ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:29.995769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27a623-0669-4c7b-af56-0a0183fe9ce7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:30.027557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 926/1843] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:30.041919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45876c-b263-4903-9b71-b12751a3a77d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:30.088946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45876c-b263-4903-9b71-b12751a3a77d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:30.119226 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85a85c-23ed-4dbf-9e8c-749284dead71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:30.121625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85a85c-23ed-4dbf-9e8c-749284dead71 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:30.121625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85a85c-23ed-4dbf-9e8c-749284dead71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:30.122810 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85a85c-23ed-4dbf-9e8c-749284dead71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:30.138190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85a85c-23ed-4dbf-9e8c-749284dead71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:30.138190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85a85c-23ed-4dbf-9e8c-749284dead71 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:30.197827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 918/1844] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:30.202014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27a623-0669-4c7b-af56-0a0183fe9ce7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:30.218580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27a623-0669-4c7b-af56-0a0183fe9ce7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:30.234796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b1d41-8bc9-4352-85de-9ffb85be2bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:30.235919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b1d41-8bc9-4352-85de-9ffb85be2bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:30.236414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b1d41-8bc9-4352-85de-9ffb85be2bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:30.236880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b1d41-8bc9-4352-85de-9ffb85be2bc9 admin admin] PATH_INFO: `/v3/projects/b157a14808f24a72a810bd87285b9212/users/56ab914418714e69960964a05d00fb3a/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:30.270561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b1d41-8bc9-4352-85de-9ffb85be2bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b157a14808f24a72a810bd87285b9212, user_id=56ab914418714e69960964a05d00fb3a, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:30.273967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b1d41-8bc9-4352-85de-9ffb85be2bc9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:30.424703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 927/1845] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:30 2022] PUT /identity/v3/projects/b157a14808f24a72a810bd87285b9212/users/56ab914418714e69960964a05d00fb3a/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:30.438691 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e5494-0048-4009-b64b-a59c3a109b65 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:30.439530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e5494-0048-4009-b64b-a59c3a109b65 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:30.441978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e5494-0048-4009-b64b-a59c3a109b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:30.548310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-de5e5494-0048-4009-b64b-a59c3a109b65 None None] MFA Rules not processed for user `56ab914418714e69960964a05d00fb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:30.550723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-de5e5494-0048-4009-b64b-a59c3a109b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:30.746287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 919/1846] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:47:30.751099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b1d41-8bc9-4352-85de-9ffb85be2bc9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:30.841116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b1d41-8bc9-4352-85de-9ffb85be2bc9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:30.846646 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a832c5e-e4ee-427b-a338-d7d01a6033b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:30.847291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a832c5e-e4ee-427b-a338-d7d01a6033b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:30.847575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a832c5e-e4ee-427b-a338-d7d01a6033b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:30.847851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a832c5e-e4ee-427b-a338-d7d01a6033b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:30.848532 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a832c5e-e4ee-427b-a338-d7d01a6033b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:30.850562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a832c5e-e4ee-427b-a338-d7d01a6033b4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:30.900918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:30.976374 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 928/1847] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:30 2022] POST /identity/v3/projects => generated 367 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:30.981563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e5494-0048-4009-b64b-a59c3a109b65 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:30.993370 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e5494-0048-4009-b64b-a59c3a109b65 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:31.058829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b91a8f-d58f-49ef-9a72-5bcf7760da8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:31.059507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b91a8f-d58f-49ef-9a72-5bcf7760da8b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:31.059719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b91a8f-d58f-49ef-9a72-5bcf7760da8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:31.059986 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b91a8f-d58f-49ef-9a72-5bcf7760da8b admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:31.060663 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b91a8f-d58f-49ef-9a72-5bcf7760da8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:31.062742 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b91a8f-d58f-49ef-9a72-5bcf7760da8b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:31.111698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:31.191030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 920/1848] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:30 2022] POST /identity/v3/users => generated 290 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:31.195519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a832c5e-e4ee-427b-a338-d7d01a6033b4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:31.210481 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a832c5e-e4ee-427b-a338-d7d01a6033b4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:31.218537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b303e49-7c51-42b9-a2b6-257f90197f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:31.219689 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b303e49-7c51-42b9-a2b6-257f90197f86 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:31.220242 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b303e49-7c51-42b9-a2b6-257f90197f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:31.220684 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b303e49-7c51-42b9-a2b6-257f90197f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:31.221865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b303e49-7c51-42b9-a2b6-257f90197f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:31.226129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b91a8f-d58f-49ef-9a72-5bcf7760da8b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:31.229962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b303e49-7c51-42b9-a2b6-257f90197f86 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:31.230127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-46b91a8f-d58f-49ef-9a72-5bcf7760da8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:31.242174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b91a8f-d58f-49ef-9a72-5bcf7760da8b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:31.247778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b9b6f-7d1d-4256-8c87-fc4ba5934324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:31.248192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 929/1849] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:31.248443 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b9b6f-7d1d-4256-8c87-fc4ba5934324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:31.248703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b9b6f-7d1d-4256-8c87-fc4ba5934324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:31.248971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b9b6f-7d1d-4256-8c87-fc4ba5934324 admin admin] PATH_INFO: `/v3/users/ddf6d219a5014b98abcb0f2b34341a3b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:31.250288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b9b6f-7d1d-4256-8c87-fc4ba5934324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddf6d219a5014b98abcb0f2b34341a3b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:31.252278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b9b6f-7d1d-4256-8c87-fc4ba5934324 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:31.253889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b303e49-7c51-42b9-a2b6-257f90197f86 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:31.269016 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b303e49-7c51-42b9-a2b6-257f90197f86 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:31.276612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b919d6-b7c9-4ed8-b51f-3578cec13a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:31.277529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b919d6-b7c9-4ed8-b51f-3578cec13a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:31.277969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b919d6-b7c9-4ed8-b51f-3578cec13a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:31.278374 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b919d6-b7c9-4ed8-b51f-3578cec13a5a admin admin] PATH_INFO: `/v3/projects/193804a0a9ab499d82bb78bdc58d9227/users/284835da9a5e4035bc1b2d1011c22d63/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:31.298727 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b919d6-b7c9-4ed8-b51f-3578cec13a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=193804a0a9ab499d82bb78bdc58d9227, user_id=284835da9a5e4035bc1b2d1011c22d63, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:31.301944 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b919d6-b7c9-4ed8-b51f-3578cec13a5a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:31.409705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 930/1850] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:31 2022] PUT /identity/v3/projects/193804a0a9ab499d82bb78bdc58d9227/users/284835da9a5e4035bc1b2d1011c22d63/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:31.417026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b919d6-b7c9-4ed8-b51f-3578cec13a5a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:31.585271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b84b9b6f-7d1d-4256-8c87-fc4ba5934324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddf6d219a5014b98abcb0f2b34341a3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:31.620260 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:31.654403 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b84b9b6f-7d1d-4256-8c87-fc4ba5934324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddf6d219a5014b98abcb0f2b34341a3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:31.678919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b919d6-b7c9-4ed8-b51f-3578cec13a5a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:31.703593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cdba8-ccdf-4f53-ac05-623700b70b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:31.717585 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cdba8-ccdf-4f53-ac05-623700b70b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:31.721186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cdba8-ccdf-4f53-ac05-623700b70b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:31.722104 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cdba8-ccdf-4f53-ac05-623700b70b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:31.726545 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cdba8-ccdf-4f53-ac05-623700b70b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:31.733273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cdba8-ccdf-4f53-ac05-623700b70b04 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:31.780491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b84b9b6f-7d1d-4256-8c87-fc4ba5934324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddf6d219a5014b98abcb0f2b34341a3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:31.785864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b84b9b6f-7d1d-4256-8c87-fc4ba5934324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddf6d219a5014b98abcb0f2b34341a3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:31.823535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 931/1851] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:47:31.828216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cdba8-ccdf-4f53-ac05-623700b70b04 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:31.830305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b84b9b6f-7d1d-4256-8c87-fc4ba5934324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddf6d219a5014b98abcb0f2b34341a3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:31.834314 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-676cdba8-ccdf-4f53-ac05-623700b70b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:31.856909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 921/1852] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:31 2022] DELETE /identity/v3/users/ddf6d219a5014b98abcb0f2b34341a3b => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:47:31.866727 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2eb695-1d11-493b-9a11-09bfa572a98d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:31.867220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2eb695-1d11-493b-9a11-09bfa572a98d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:31.874266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2eb695-1d11-493b-9a11-09bfa572a98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:31.896100 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cdba8-ccdf-4f53-ac05-623700b70b04 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:31.947379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-de2eb695-1d11-493b-9a11-09bfa572a98d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:31.953125 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-de2eb695-1d11-493b-9a11-09bfa572a98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:32.542888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea2fe0-7ebc-486a-bb16-30c0a1e0968b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:32.544327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea2fe0-7ebc-486a-bb16-30c0a1e0968b admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:32.544695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea2fe0-7ebc-486a-bb16-30c0a1e0968b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:32.545193 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea2fe0-7ebc-486a-bb16-30c0a1e0968b admin admin] PATH_INFO: `/v3/projects/193804a0a9ab499d82bb78bdc58d9227/users/284835da9a5e4035bc1b2d1011c22d63/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:32.549104 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea2fe0-7ebc-486a-bb16-30c0a1e0968b admin admin] RBAC: Authorizing `identity:create_grant(project_id=193804a0a9ab499d82bb78bdc58d9227, user_id=284835da9a5e4035bc1b2d1011c22d63, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:32.555631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea2fe0-7ebc-486a-bb16-30c0a1e0968b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:32.558890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 922/1853] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 697 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:32.575779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2eb695-1d11-493b-9a11-09bfa572a98d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:32.577599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-de2eb695-1d11-493b-9a11-09bfa572a98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:32.577885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 932/1854] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:31 2022] PUT /identity/v3/projects/193804a0a9ab499d82bb78bdc58d9227/users/284835da9a5e4035bc1b2d1011c22d63/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:32.598947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a33fcc-11b0-42c4-b91b-1434415b4ea0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:32.599608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a33fcc-11b0-42c4-b91b-1434415b4ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:32.601545 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a33fcc-11b0-42c4-b91b-1434415b4ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:32.679564 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c3a33fcc-11b0-42c4-b91b-1434415b4ea0 None None] MFA Rules not processed for user `284835da9a5e4035bc1b2d1011c22d63`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:32.681716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2eb695-1d11-493b-9a11-09bfa572a98d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:32.682125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c3a33fcc-11b0-42c4-b91b-1434415b4ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:32.762282 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae36a9-aeb7-415a-9a27-3d7e5eba82d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:32.763486 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae36a9-aeb7-415a-9a27-3d7e5eba82d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:32.764290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae36a9-aeb7-415a-9a27-3d7e5eba82d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:32.764862 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae36a9-aeb7-415a-9a27-3d7e5eba82d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:32.830838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 933/1855] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:47:32.859898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae36a9-aeb7-415a-9a27-3d7e5eba82d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:32.860300 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae36a9-aeb7-415a-9a27-3d7e5eba82d9 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:32.942578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 923/1856] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:33.552815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:33.751458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a33fcc-11b0-42c4-b91b-1434415b4ea0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:33.754317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c3a33fcc-11b0-42c4-b91b-1434415b4ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:33.757453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:33.905972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a33fcc-11b0-42c4-b91b-1434415b4ea0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:33.910234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c2dc1-d32a-4bef-b3c4-854bafa72751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:33.911311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c2dc1-d32a-4bef-b3c4-854bafa72751 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:33.911693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c2dc1-d32a-4bef-b3c4-854bafa72751 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:33.912162 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c2dc1-d32a-4bef-b3c4-854bafa72751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:33.924524 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c2dc1-d32a-4bef-b3c4-854bafa72751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:33.927249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c2dc1-d32a-4bef-b3c4-854bafa72751 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:33.946496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 934/1857] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:34.002678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae36a9-aeb7-415a-9a27-3d7e5eba82d9 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:34.005786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-55ae36a9-aeb7-415a-9a27-3d7e5eba82d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:34.067181 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae36a9-aeb7-415a-9a27-3d7e5eba82d9 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:34.145762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:34.145762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:34.145762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:34.145762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] PATH_INFO: `/v3/projects/b1f0f7cba1ac487094c99454bb57e792` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:34.146503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1f0f7cba1ac487094c99454bb57e792)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:34.146503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:34.499308 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1f0f7cba1ac487094c99454bb57e792', 'request_id': 'req-551bb144-021f-496f-836d-cfb757d0487b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:34.499640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c2dc1-d32a-4bef-b3c4-854bafa72751 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:34.503162 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-411c2dc1-d32a-4bef-b3c4-854bafa72751 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:34.550686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c2dc1-d32a-4bef-b3c4-854bafa72751 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:34.601443 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1f0f7cba1ac487094c99454bb57e792', 'request_id': 'req-551bb144-021f-496f-836d-cfb757d0487b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:34.643730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1f0f7cba1ac487094c99454bb57e792', 'request_id': 'req-551bb144-021f-496f-836d-cfb757d0487b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:34.665013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] The token cache is being invalidate because project b1f0f7cba1ac487094c99454bb57e792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:34.667317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:34.668433 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:34.671507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 924/1858] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:34 2022] DELETE /identity/v3/projects/b1f0f7cba1ac487094c99454bb57e792 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:34.677565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:34.680721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:34.794051 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5431fde-e24f-44cb-ad37-7cb9443f4bbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:34.796856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5431fde-e24f-44cb-ad37-7cb9443f4bbf service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:34.797539 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5431fde-e24f-44cb-ad37-7cb9443f4bbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:34.797826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5431fde-e24f-44cb-ad37-7cb9443f4bbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:34.804378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a5431fde-e24f-44cb-ad37-7cb9443f4bbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:34.843006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bb144-021f-496f-836d-cfb757d0487b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:34.868560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ff253-2c65-4ad9-91b8-9117d2321c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:34.869601 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ff253-2c65-4ad9-91b8-9117d2321c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:34.869886 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ff253-2c65-4ad9-91b8-9117d2321c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:34.870540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ff253-2c65-4ad9-91b8-9117d2321c4c admin admin] PATH_INFO: `/v3/users/174e02b658d94a4fab561fb4a4a8fa5e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:34.876625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ff253-2c65-4ad9-91b8-9117d2321c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=174e02b658d94a4fab561fb4a4a8fa5e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:34.885509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ff253-2c65-4ad9-91b8-9117d2321c4c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:35.004260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5431fde-e24f-44cb-ad37-7cb9443f4bbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:35.010180 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5431fde-e24f-44cb-ad37-7cb9443f4bbf service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:35.042239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 935/1859] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:47:35.202769 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0b1ff253-2c65-4ad9-91b8-9117d2321c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '174e02b658d94a4fab561fb4a4a8fa5e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:35.257661 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0b1ff253-2c65-4ad9-91b8-9117d2321c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '174e02b658d94a4fab561fb4a4a8fa5e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:35.332212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0b1ff253-2c65-4ad9-91b8-9117d2321c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '174e02b658d94a4fab561fb4a4a8fa5e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:35.332998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0b1ff253-2c65-4ad9-91b8-9117d2321c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '174e02b658d94a4fab561fb4a4a8fa5e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:35.351024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0b1ff253-2c65-4ad9-91b8-9117d2321c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '174e02b658d94a4fab561fb4a4a8fa5e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:35.386885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 925/1860] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:34 2022] DELETE /identity/v3/users/174e02b658d94a4fab561fb4a4a8fa5e => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:36.002207 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:36.426877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5431fde-e24f-44cb-ad37-7cb9443f4bbf service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:36.449283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5431fde-e24f-44cb-ad37-7cb9443f4bbf service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:36.459902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:36.626745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:36.628879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:36.629591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:36.630369 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8 admin admin] PATH_INFO: `/v3/projects/d2ad4b447fc14e9a993abc9b937439b6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:36.633192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ad4b447fc14e9a993abc9b937439b6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:36.636614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:36.818936 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ad4b447fc14e9a993abc9b937439b6', 'request_id': 'req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:36.901134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ad4b447fc14e9a993abc9b937439b6', 'request_id': 'req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:36.913320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ad4b447fc14e9a993abc9b937439b6', 'request_id': 'req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:36.937624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8 admin admin] The token cache is being invalidate because project d2ad4b447fc14e9a993abc9b937439b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:36.938361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:36.939591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bc422d79-3d62-47c3-bddd-d4f318d6c7d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:36.942359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 936/1861] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:36 2022] DELETE /identity/v3/projects/d2ad4b447fc14e9a993abc9b937439b6 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:36.951309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a940e3-d4cf-4373-8cda-a4890e60d373 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:36.955510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a940e3-d4cf-4373-8cda-a4890e60d373 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:36.955510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a940e3-d4cf-4373-8cda-a4890e60d373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:36.993012 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-43a940e3-d4cf-4373-8cda-a4890e60d373 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:36.996336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-43a940e3-d4cf-4373-8cda-a4890e60d373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:37.077148 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 926/1862] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:37.088534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5a78-025b-4356-a80d-ebc1909cb292 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:37.089356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5a78-025b-4356-a80d-ebc1909cb292 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:37.089839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5a78-025b-4356-a80d-ebc1909cb292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:37.122846 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a85a5a78-025b-4356-a80d-ebc1909cb292 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:37.133353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a85a5a78-025b-4356-a80d-ebc1909cb292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:37.139962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 937/1863] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:37.142362 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a940e3-d4cf-4373-8cda-a4890e60d373 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:37.157484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a940e3-d4cf-4373-8cda-a4890e60d373 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:37.164076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87532dab-a056-4f85-a18a-7c13299b7dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:37.164733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87532dab-a056-4f85-a18a-7c13299b7dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:37.165037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87532dab-a056-4f85-a18a-7c13299b7dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:37.165384 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87532dab-a056-4f85-a18a-7c13299b7dd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:37.166274 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87532dab-a056-4f85-a18a-7c13299b7dd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:47:37.166606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87532dab-a056-4f85-a18a-7c13299b7dd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:37.168385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87532dab-a056-4f85-a18a-7c13299b7dd1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:37.185348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 927/1864] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:47:37.192538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a5a78-025b-4356-a80d-ebc1909cb292 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:37.213424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a5a78-025b-4356-a80d-ebc1909cb292 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:37.222695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1dc5f-1fbe-418d-b2fc-124e99bf3d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:37.223840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1dc5f-1fbe-418d-b2fc-124e99bf3d02 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:37.224412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1dc5f-1fbe-418d-b2fc-124e99bf3d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:37.224945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1dc5f-1fbe-418d-b2fc-124e99bf3d02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:37.226383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1dc5f-1fbe-418d-b2fc-124e99bf3d02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:37.231347 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1dc5f-1fbe-418d-b2fc-124e99bf3d02 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:37.383594 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 938/1865] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:37 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:37.391957 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87532dab-a056-4f85-a18a-7c13299b7dd1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:37.415959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87532dab-a056-4f85-a18a-7c13299b7dd1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:37.549502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8721f-b480-4b18-ac59-0cff8376f93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:37.550613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8721f-b480-4b18-ac59-0cff8376f93a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:37.551071 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8721f-b480-4b18-ac59-0cff8376f93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:37.551552 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8721f-b480-4b18-ac59-0cff8376f93a admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:37.554379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8721f-b480-4b18-ac59-0cff8376f93a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:37.557258 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8721f-b480-4b18-ac59-0cff8376f93a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:37.576032 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:37.723541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 928/1866] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:37 2022] POST /identity/v3/users => generated 290 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:47:37.725143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1dc5f-1fbe-418d-b2fc-124e99bf3d02 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:37.738047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1dc5f-1fbe-418d-b2fc-124e99bf3d02 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:37.743109 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493b79a-195b-478f-8df8-9869ced6a01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:37.743872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493b79a-195b-478f-8df8-9869ced6a01b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:37.744287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493b79a-195b-478f-8df8-9869ced6a01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:37.744557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493b79a-195b-478f-8df8-9869ced6a01b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:37.745223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493b79a-195b-478f-8df8-9869ced6a01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:37.746902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493b79a-195b-478f-8df8-9869ced6a01b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:37.760864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 939/1867] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:37.767335 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8721f-b480-4b18-ac59-0cff8376f93a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:37.784429 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8721f-b480-4b18-ac59-0cff8376f93a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:37.798299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d1841-2f45-4897-8954-40c103ed82ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:37.800310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d1841-2f45-4897-8954-40c103ed82ee admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:37.800310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d1841-2f45-4897-8954-40c103ed82ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:37.800926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d1841-2f45-4897-8954-40c103ed82ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:37.802307 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d1841-2f45-4897-8954-40c103ed82ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:37.805901 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d1841-2f45-4897-8954-40c103ed82ee admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:37.838337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 929/1868] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:37.844948 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493b79a-195b-478f-8df8-9869ced6a01b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:37.867922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493b79a-195b-478f-8df8-9869ced6a01b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:37.876889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54abe4-844d-4e4a-babe-45e5717dfe7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:37.878305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54abe4-844d-4e4a-babe-45e5717dfe7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:37.879469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54abe4-844d-4e4a-babe-45e5717dfe7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:37.880619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54abe4-844d-4e4a-babe-45e5717dfe7a admin admin] PATH_INFO: `/v3/projects/b6cb73575c1a48f181139dbcf2cdfd68/users/666857e8209e4e72b3f998e4077b3204/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:37.910415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54abe4-844d-4e4a-babe-45e5717dfe7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6cb73575c1a48f181139dbcf2cdfd68, user_id=666857e8209e4e72b3f998e4077b3204, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:37.914327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54abe4-844d-4e4a-babe-45e5717dfe7a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:38.020337 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 940/1869] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:37 2022] PUT /identity/v3/projects/b6cb73575c1a48f181139dbcf2cdfd68/users/666857e8209e4e72b3f998e4077b3204/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:38.040278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92334dc-610f-4649-b137-ed89b4001578 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:38.040920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92334dc-610f-4649-b137-ed89b4001578 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:38.041357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92334dc-610f-4649-b137-ed89b4001578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:38.112363 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d92334dc-610f-4649-b137-ed89b4001578 None None] MFA Rules not processed for user `666857e8209e4e72b3f998e4077b3204`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:38.116675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d92334dc-610f-4649-b137-ed89b4001578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:38.236630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 930/1870] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:47:38.253061 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54abe4-844d-4e4a-babe-45e5717dfe7a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:38.369870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:38.407609 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54abe4-844d-4e4a-babe-45e5717dfe7a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:38.430923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b99d96-5b0d-4119-9bdc-3ecca21832b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:38.432531 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b99d96-5b0d-4119-9bdc-3ecca21832b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:38.433013 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b99d96-5b0d-4119-9bdc-3ecca21832b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:38.433488 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b99d96-5b0d-4119-9bdc-3ecca21832b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:38.434703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b99d96-5b0d-4119-9bdc-3ecca21832b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:38.438625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b99d96-5b0d-4119-9bdc-3ecca21832b4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:38.565628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 941/1871] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:38 2022] POST /identity/v3/projects => generated 377 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:38.572464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92334dc-610f-4649-b137-ed89b4001578 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:38.591990 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92334dc-610f-4649-b137-ed89b4001578 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:38.760076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56a1a7-bb40-4870-bbe6-1909db273f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:38.762270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56a1a7-bb40-4870-bbe6-1909db273f27 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:38.762270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56a1a7-bb40-4870-bbe6-1909db273f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:38.762731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56a1a7-bb40-4870-bbe6-1909db273f27 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:38.763922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a56a1a7-bb40-4870-bbe6-1909db273f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:38.770562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a56a1a7-bb40-4870-bbe6-1909db273f27 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:38.991128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 931/1872] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:38 2022] POST /identity/v3/users => generated 289 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:39.000290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b99d96-5b0d-4119-9bdc-3ecca21832b4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:39.023127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b99d96-5b0d-4119-9bdc-3ecca21832b4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:39.033177 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e9050-dd43-4e25-b461-5fa521dfa10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:39.034577 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e9050-dd43-4e25-b461-5fa521dfa10b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:39.034577 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e9050-dd43-4e25-b461-5fa521dfa10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:39.034903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e9050-dd43-4e25-b461-5fa521dfa10b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:39.035432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e9050-dd43-4e25-b461-5fa521dfa10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:39.037101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e9050-dd43-4e25-b461-5fa521dfa10b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:39.053625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 942/1873] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:39.060748 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56a1a7-bb40-4870-bbe6-1909db273f27 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:39.076498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56a1a7-bb40-4870-bbe6-1909db273f27 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:39.083394 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428129d-c047-4c08-9e05-1f0c931df8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:39.084199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428129d-c047-4c08-9e05-1f0c931df8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:39.084584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428129d-c047-4c08-9e05-1f0c931df8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:39.085073 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428129d-c047-4c08-9e05-1f0c931df8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:39.085838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428129d-c047-4c08-9e05-1f0c931df8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:39.088388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428129d-c047-4c08-9e05-1f0c931df8b4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:39.103841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 932/1874] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:39.109101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e9050-dd43-4e25-b461-5fa521dfa10b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:39.121415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e9050-dd43-4e25-b461-5fa521dfa10b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:39.126631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4215d2-ab46-44df-bf5a-19cffb590584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:39.127345 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4215d2-ab46-44df-bf5a-19cffb590584 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:39.127920 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4215d2-ab46-44df-bf5a-19cffb590584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:39.127920 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4215d2-ab46-44df-bf5a-19cffb590584 admin admin] PATH_INFO: `/v3/projects/216cead48398453091d419dead701e31/users/6782e09cfa2d471b8da4736e564f1256/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:39.146381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4215d2-ab46-44df-bf5a-19cffb590584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=216cead48398453091d419dead701e31, user_id=6782e09cfa2d471b8da4736e564f1256, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:39.149230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4215d2-ab46-44df-bf5a-19cffb590584 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:39.256232 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 943/1875] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:39 2022] PUT /identity/v3/projects/216cead48398453091d419dead701e31/users/6782e09cfa2d471b8da4736e564f1256/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:47:39.269493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4ca67-c380-442a-9677-8b0f6af5c286 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:39.270542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4ca67-c380-442a-9677-8b0f6af5c286 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:39.271048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4ca67-c380-442a-9677-8b0f6af5c286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:39.315352 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2be4ca67-c380-442a-9677-8b0f6af5c286 None None] MFA Rules not processed for user `6782e09cfa2d471b8da4736e564f1256`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:39.315988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2be4ca67-c380-442a-9677-8b0f6af5c286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:39.392181 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:39.437170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 933/1876] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:39.444342 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4215d2-ab46-44df-bf5a-19cffb590584 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:39.600857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4215d2-ab46-44df-bf5a-19cffb590584 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:39.611087 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6ee89-3542-4755-9694-f73ce55aefeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:39.612595 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6ee89-3542-4755-9694-f73ce55aefeb admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:39.613086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6ee89-3542-4755-9694-f73ce55aefeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:39.613624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6ee89-3542-4755-9694-f73ce55aefeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:39.614840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6ee89-3542-4755-9694-f73ce55aefeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:39.618697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6ee89-3542-4755-9694-f73ce55aefeb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:39.808314 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 944/1877] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:39 2022] POST /identity/v3/projects => generated 379 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:39.812173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4ca67-c380-442a-9677-8b0f6af5c286 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:39.831386 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4ca67-c380-442a-9677-8b0f6af5c286 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:39.953358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9bf06-a1bc-4ea5-a75b-bf2a1b30d77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:39.954455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9bf06-a1bc-4ea5-a75b-bf2a1b30d77a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:39.955048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9bf06-a1bc-4ea5-a75b-bf2a1b30d77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:39.955560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9bf06-a1bc-4ea5-a75b-bf2a1b30d77a admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:39.956773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9bf06-a1bc-4ea5-a75b-bf2a1b30d77a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:39.960902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9bf06-a1bc-4ea5-a75b-bf2a1b30d77a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:40.066769 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 934/1878] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:39 2022] POST /identity/v3/users => generated 296 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:40.072780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6ee89-3542-4755-9694-f73ce55aefeb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:40.086099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6ee89-3542-4755-9694-f73ce55aefeb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:40.092472 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf9542-fe2a-45eb-9344-d58b8d4a61de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:40.093212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf9542-fe2a-45eb-9344-d58b8d4a61de admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:40.094379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf9542-fe2a-45eb-9344-d58b8d4a61de admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:40.094379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf9542-fe2a-45eb-9344-d58b8d4a61de admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:40.094968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf9542-fe2a-45eb-9344-d58b8d4a61de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:40.096980 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf9542-fe2a-45eb-9344-d58b8d4a61de admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:40.114636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 945/1879] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:47:40.120918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9bf06-a1bc-4ea5-a75b-bf2a1b30d77a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:40.134805 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9bf06-a1bc-4ea5-a75b-bf2a1b30d77a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:40.140078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb402d84-2903-4c6c-a064-87d0fa5329d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:40.140754 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb402d84-2903-4c6c-a064-87d0fa5329d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:40.141030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb402d84-2903-4c6c-a064-87d0fa5329d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:40.141302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb402d84-2903-4c6c-a064-87d0fa5329d9 admin admin] PATH_INFO: `/v3/projects/8e13fd935a2b4c01a2403f91c8eb818a/users/6aa2ea9356f44fc286f3bee2d28eb722/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:40.161562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb402d84-2903-4c6c-a064-87d0fa5329d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e13fd935a2b4c01a2403f91c8eb818a, user_id=6aa2ea9356f44fc286f3bee2d28eb722, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:40.166365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb402d84-2903-4c6c-a064-87d0fa5329d9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:40.301541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 935/1880] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:40 2022] PUT /identity/v3/projects/8e13fd935a2b4c01a2403f91c8eb818a/users/6aa2ea9356f44fc286f3bee2d28eb722/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:47:40.306658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf9542-fe2a-45eb-9344-d58b8d4a61de admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:40.318717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf9542-fe2a-45eb-9344-d58b8d4a61de admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:40.391932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5784f-5bef-4a9c-a8fc-a508ad7f526b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:40.392695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5784f-5bef-4a9c-a8fc-a508ad7f526b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:40.393099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5784f-5bef-4a9c-a8fc-a508ad7f526b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:40.400023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5784f-5bef-4a9c-a8fc-a508ad7f526b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:40.400023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5784f-5bef-4a9c-a8fc-a508ad7f526b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:40.400023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5784f-5bef-4a9c-a8fc-a508ad7f526b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:40.400023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:40.412128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 946/1881] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:40.416991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb402d84-2903-4c6c-a064-87d0fa5329d9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:40.429545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb402d84-2903-4c6c-a064-87d0fa5329d9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:40.434853 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e64b8a-6efd-428f-b663-71ab3307d9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:40.435542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e64b8a-6efd-428f-b663-71ab3307d9a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:40.439518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e64b8a-6efd-428f-b663-71ab3307d9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:40.439518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e64b8a-6efd-428f-b663-71ab3307d9a1 admin admin] PATH_INFO: `/v3/projects/8e13fd935a2b4c01a2403f91c8eb818a/users/6aa2ea9356f44fc286f3bee2d28eb722/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:40.439518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e64b8a-6efd-428f-b663-71ab3307d9a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e13fd935a2b4c01a2403f91c8eb818a, user_id=6aa2ea9356f44fc286f3bee2d28eb722, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:40.441558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e64b8a-6efd-428f-b663-71ab3307d9a1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:40.453599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 936/1882] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:40 2022] PUT /identity/v3/projects/8e13fd935a2b4c01a2403f91c8eb818a/users/6aa2ea9356f44fc286f3bee2d28eb722/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:40.465921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cefd5-2006-4acd-bb9a-12ca9824ed42 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:40.466243 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cefd5-2006-4acd-bb9a-12ca9824ed42 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:40.466589 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cefd5-2006-4acd-bb9a-12ca9824ed42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:40.498876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e19cefd5-2006-4acd-bb9a-12ca9824ed42 None None] MFA Rules not processed for user `6aa2ea9356f44fc286f3bee2d28eb722`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:40.500132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e19cefd5-2006-4acd-bb9a-12ca9824ed42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:40.591235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 947/1883] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:40.603920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e64b8a-6efd-428f-b663-71ab3307d9a1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:40.606056 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a6e64b8a-6efd-428f-b663-71ab3307d9a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:40.723226 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e64b8a-6efd-428f-b663-71ab3307d9a1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:40.736357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13989984-def5-43a2-8957-7e2d02419d57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:40.738058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13989984-def5-43a2-8957-7e2d02419d57 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:40.738825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13989984-def5-43a2-8957-7e2d02419d57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:40.739544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13989984-def5-43a2-8957-7e2d02419d57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:40.758441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13989984-def5-43a2-8957-7e2d02419d57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:40.762308 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13989984-def5-43a2-8957-7e2d02419d57 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:40.815828 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 937/1884] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:47:41.112390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cefd5-2006-4acd-bb9a-12ca9824ed42 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:41.115643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e19cefd5-2006-4acd-bb9a-12ca9824ed42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:41.135525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cefd5-2006-4acd-bb9a-12ca9824ed42 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:41.142833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6c688-76cd-44e0-bab6-0cd259b44859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:41.144406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6c688-76cd-44e0-bab6-0cd259b44859 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:41.145540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6c688-76cd-44e0-bab6-0cd259b44859 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:41.146248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6c688-76cd-44e0-bab6-0cd259b44859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:41.237046 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6c688-76cd-44e0-bab6-0cd259b44859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:41.238617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6c688-76cd-44e0-bab6-0cd259b44859 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:41.250641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 948/1885] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:42.239032 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33429691-d169-48e3-ac5d-bb111d17dfcb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:42.239519 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33429691-d169-48e3-ac5d-bb111d17dfcb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:42.246674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33429691-d169-48e3-ac5d-bb111d17dfcb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:42.246674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 938/1886] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:47:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:47:42.278015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb5b24-0c07-414f-857c-988290a44b47 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:42.278599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb5b24-0c07-414f-857c-988290a44b47 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:42.278797 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb5b24-0c07-414f-857c-988290a44b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:42.315438 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ffeb5b24-0c07-414f-857c-988290a44b47 None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:42.318744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ffeb5b24-0c07-414f-857c-988290a44b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:42.487679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 949/1887] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:47:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:47:42.578337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33429691-d169-48e3-ac5d-bb111d17dfcb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:42.612138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33429691-d169-48e3-ac5d-bb111d17dfcb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:42.615166 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb5b24-0c07-414f-857c-988290a44b47 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:42.617549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ffeb5b24-0c07-414f-857c-988290a44b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:42.624894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a798d-6fef-4ef3-a722-c45ba63fa656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:42.627243 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a798d-6fef-4ef3-a722-c45ba63fa656 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:42.628063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a798d-6fef-4ef3-a722-c45ba63fa656 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:42.628769 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a798d-6fef-4ef3-a722-c45ba63fa656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:42.634812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fc5a798d-6fef-4ef3-a722-c45ba63fa656 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:42.670214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb5b24-0c07-414f-857c-988290a44b47 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:42.689672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415d3b3-17e4-43df-b012-6b8df7709d1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:42.691031 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415d3b3-17e4-43df-b012-6b8df7709d1c service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:42.691655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415d3b3-17e4-43df-b012-6b8df7709d1c service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:42.692371 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415d3b3-17e4-43df-b012-6b8df7709d1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:42.719093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415d3b3-17e4-43df-b012-6b8df7709d1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:42.723572 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415d3b3-17e4-43df-b012-6b8df7709d1c service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:42.741388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:42.758410 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:42.761503 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 950/1888] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:47:42.882614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a798d-6fef-4ef3-a722-c45ba63fa656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:42.884345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a798d-6fef-4ef3-a722-c45ba63fa656 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:42.914539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 939/1889] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:47.517569 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415d3b3-17e4-43df-b012-6b8df7709d1c service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:47.651296 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415d3b3-17e4-43df-b012-6b8df7709d1c service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:47.660093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:47.661163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:47.661622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:47.667214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] PATH_INFO: `/v3/users/666857e8209e4e72b3f998e4077b3204` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:47.667214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] RBAC: Authorizing `identity:delete_user(user_id=666857e8209e4e72b3f998e4077b3204)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:47.668597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:47.975716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '666857e8209e4e72b3f998e4077b3204'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:47.975716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '666857e8209e4e72b3f998e4077b3204'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:48.107023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '666857e8209e4e72b3f998e4077b3204'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:48.125745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '666857e8209e4e72b3f998e4077b3204'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:48.142285 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '666857e8209e4e72b3f998e4077b3204'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:48.158494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 951/1890] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:47 2022] DELETE /identity/v3/users/666857e8209e4e72b3f998e4077b3204 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:48.161470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38eebf7-aee7-4eac-a99b-0e4135d73c88 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:48.162120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38eebf7-aee7-4eac-a99b-0e4135d73c88 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:48.162654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38eebf7-aee7-4eac-a99b-0e4135d73c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:48.204661 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b38eebf7-aee7-4eac-a99b-0e4135d73c88 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:48.206853 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b38eebf7-aee7-4eac-a99b-0e4135d73c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:48.365262 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 940/1891] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:48.377801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:48.380143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:48.511867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bbbab-6b6b-4542-bfa5-f674e1d0393b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:48.519196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b654611-d040-4b98-b7db-4200847a4a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:48.520347 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b654611-d040-4b98-b7db-4200847a4a88 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:48.520790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b654611-d040-4b98-b7db-4200847a4a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:48.521279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b654611-d040-4b98-b7db-4200847a4a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:48.540674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b654611-d040-4b98-b7db-4200847a4a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:48.543429 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b654611-d040-4b98-b7db-4200847a4a88 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:48.561493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 952/1892] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:49.507020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38eebf7-aee7-4eac-a99b-0e4135d73c88 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:49.510883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b38eebf7-aee7-4eac-a99b-0e4135d73c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:49.547621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38eebf7-aee7-4eac-a99b-0e4135d73c88 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:49.562768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:49.564211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:49.564795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:49.565501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] PATH_INFO: `/v3/projects/b6cb73575c1a48f181139dbcf2cdfd68` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:49.567798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6cb73575c1a48f181139dbcf2cdfd68)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:49.590567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:49.693199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b654611-d040-4b98-b7db-4200847a4a88 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:49.694872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8b654611-d040-4b98-b7db-4200847a4a88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:49.716319 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b654611-d040-4b98-b7db-4200847a4a88 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:49.726385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0a2d3-1e5b-4f28-a72b-c87fbe07f0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:49.727811 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0a2d3-1e5b-4f28-a72b-c87fbe07f0fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:49.728461 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0a2d3-1e5b-4f28-a72b-c87fbe07f0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:49.729132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0a2d3-1e5b-4f28-a72b-c87fbe07f0fb admin admin] PATH_INFO: `/v3/users/56ab914418714e69960964a05d00fb3a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:49.731879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0a2d3-1e5b-4f28-a72b-c87fbe07f0fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=56ab914418714e69960964a05d00fb3a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:49.737808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0a2d3-1e5b-4f28-a72b-c87fbe07f0fb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:49.775563 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:49.788300 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6cb73575c1a48f181139dbcf2cdfd68', 'request_id': 'req-1b137581-fb17-4740-8c90-23c024d222ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:49.889783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6cb73575c1a48f181139dbcf2cdfd68', 'request_id': 'req-1b137581-fb17-4740-8c90-23c024d222ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:49.901949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6cb73575c1a48f181139dbcf2cdfd68', 'request_id': 'req-1b137581-fb17-4740-8c90-23c024d222ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:49.920886 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] The token cache is being invalidate because project b6cb73575c1a48f181139dbcf2cdfd68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:49.921333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:49.922073 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:49.923705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 941/1893] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:49 2022] DELETE /identity/v3/projects/b6cb73575c1a48f181139dbcf2cdfd68 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:49.929027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:49.931201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:49.996125 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:50.010004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b137581-fb17-4740-8c90-23c024d222ef admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:50.015182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:50.015893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:50.016206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:50.016480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] PATH_INFO: `/v3/users/6782e09cfa2d471b8da4736e564f1256` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:50.019177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6782e09cfa2d471b8da4736e564f1256)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:50.023102 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:50.079874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a6a0a2d3-1e5b-4f28-a72b-c87fbe07f0fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56ab914418714e69960964a05d00fb3a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:50.080898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a6a0a2d3-1e5b-4f28-a72b-c87fbe07f0fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56ab914418714e69960964a05d00fb3a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:50.214352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a6a0a2d3-1e5b-4f28-a72b-c87fbe07f0fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56ab914418714e69960964a05d00fb3a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:50.228514 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a6a0a2d3-1e5b-4f28-a72b-c87fbe07f0fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56ab914418714e69960964a05d00fb3a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:50.261533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a6a0a2d3-1e5b-4f28-a72b-c87fbe07f0fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56ab914418714e69960964a05d00fb3a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:50.271817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 953/1894] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:49 2022] DELETE /identity/v3/users/56ab914418714e69960964a05d00fb3a => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:47:50.277585 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3573-a53b-43e9-917a-656b36830875 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:50.277865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3573-a53b-43e9-917a-656b36830875 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:50.278127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3573-a53b-43e9-917a-656b36830875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:50.300089 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-db5b3573-a53b-43e9-917a-656b36830875 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:50.301556 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-db5b3573-a53b-43e9-917a-656b36830875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:50.343331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6782e09cfa2d471b8da4736e564f1256'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:50.358949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6782e09cfa2d471b8da4736e564f1256'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:50.378250 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 954/1895] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:50.391076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b3573-a53b-43e9-917a-656b36830875 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:50.392577 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-db5b3573-a53b-43e9-917a-656b36830875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:50.457043 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b3573-a53b-43e9-917a-656b36830875 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:50.459936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6782e09cfa2d471b8da4736e564f1256'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:50.460633 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6782e09cfa2d471b8da4736e564f1256'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:50.461689 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fb0cd-861e-44b3-a3af-a4ee5eaac894 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:50.462923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fb0cd-861e-44b3-a3af-a4ee5eaac894 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:50.463235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fb0cd-861e-44b3-a3af-a4ee5eaac894 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:50.463510 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fb0cd-861e-44b3-a3af-a4ee5eaac894 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:50.465508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-917fb0cd-861e-44b3-a3af-a4ee5eaac894 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:50.472830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6782e09cfa2d471b8da4736e564f1256'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:50.484774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 942/1896] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:49 2022] DELETE /identity/v3/users/6782e09cfa2d471b8da4736e564f1256 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:47:50.486358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917fb0cd-861e-44b3-a3af-a4ee5eaac894 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:50.489131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917fb0cd-861e-44b3-a3af-a4ee5eaac894 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:50.513753 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 955/1897] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:47:51.203985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:51.205520 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:51.242538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fb0cd-861e-44b3-a3af-a4ee5eaac894 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:51.247005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-917fb0cd-861e-44b3-a3af-a4ee5eaac894 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:51.292537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fb0cd-861e-44b3-a3af-a4ee5eaac894 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:51.324901 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d32c7-a3fd-4056-8c30-48511c91e2fc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:51.333054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:51.334279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:51.334824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:51.335326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] PATH_INFO: `/v3/users/48afe85e198748b1ac45e4f86dad33ef` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:51.338173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=48afe85e198748b1ac45e4f86dad33ef)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:51.343349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:51.349434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:51.440880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:51.441575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:51.441920 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:51.442231 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] PATH_INFO: `/v3/projects/216cead48398453091d419dead701e31` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:51.445283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=216cead48398453091d419dead701e31)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:51.448311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:51.700303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48afe85e198748b1ac45e4f86dad33ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:51.700870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '216cead48398453091d419dead701e31', 'request_id': 'req-e45a015c-72fd-468b-bee2-21571e758980'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:51.722222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48afe85e198748b1ac45e4f86dad33ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:51.888992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '216cead48398453091d419dead701e31', 'request_id': 'req-e45a015c-72fd-468b-bee2-21571e758980'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:51.904662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '216cead48398453091d419dead701e31', 'request_id': 'req-e45a015c-72fd-468b-bee2-21571e758980'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:51.920809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] The token cache is being invalidate because project 216cead48398453091d419dead701e31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:51.921014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:51.921692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:51.923251 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 956/1898] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:51 2022] DELETE /identity/v3/projects/216cead48398453091d419dead701e31 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:51.929326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:51.935123 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:51.987052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48afe85e198748b1ac45e4f86dad33ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:51.990584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48afe85e198748b1ac45e4f86dad33ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:51.996444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:52.014527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48afe85e198748b1ac45e4f86dad33ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:52.027049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 943/1899] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:51 2022] DELETE /identity/v3/users/48afe85e198748b1ac45e4f86dad33ef => generated 0 bytes in 823 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:47:52.028431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:52.031913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:52.088506 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a015c-72fd-468b-bee2-21571e758980 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:52.151940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142abd2e-884e-4026-866a-3c882f5182cb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:52.159378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:52.160397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:52.160675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:52.161134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] PATH_INFO: `/v3/users/6aa2ea9356f44fc286f3bee2d28eb722` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:52.164646 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa2ea9356f44fc286f3bee2d28eb722)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:52.164646 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:52.173851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994b016-9257-474f-9e0c-88e142537403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:52.174785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994b016-9257-474f-9e0c-88e142537403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:52.175111 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994b016-9257-474f-9e0c-88e142537403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:52.175400 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994b016-9257-474f-9e0c-88e142537403 admin admin] PATH_INFO: `/v3/projects/b157a14808f24a72a810bd87285b9212` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:52.176879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994b016-9257-474f-9e0c-88e142537403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b157a14808f24a72a810bd87285b9212)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:52.179103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994b016-9257-474f-9e0c-88e142537403 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:52.206911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:52.478121 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a994b016-9257-474f-9e0c-88e142537403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b157a14808f24a72a810bd87285b9212', 'request_id': 'req-a994b016-9257-474f-9e0c-88e142537403'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:52.479709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa2ea9356f44fc286f3bee2d28eb722'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:52.497534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa2ea9356f44fc286f3bee2d28eb722'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:52.602504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a994b016-9257-474f-9e0c-88e142537403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b157a14808f24a72a810bd87285b9212', 'request_id': 'req-a994b016-9257-474f-9e0c-88e142537403'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:52.611275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a994b016-9257-474f-9e0c-88e142537403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b157a14808f24a72a810bd87285b9212', 'request_id': 'req-a994b016-9257-474f-9e0c-88e142537403'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:52.624466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a994b016-9257-474f-9e0c-88e142537403 admin admin] The token cache is being invalidate because project b157a14808f24a72a810bd87285b9212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:52.624755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a994b016-9257-474f-9e0c-88e142537403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:52.625436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a994b016-9257-474f-9e0c-88e142537403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:52.627001 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 957/1900] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:51 2022] DELETE /identity/v3/projects/b157a14808f24a72a810bd87285b9212 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:52.628238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd0be2-9b65-4635-8734-1c4e797bfcba None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:52.628522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd0be2-9b65-4635-8734-1c4e797bfcba None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:52.628840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd0be2-9b65-4635-8734-1c4e797bfcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:52.649733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-28dd0be2-9b65-4635-8734-1c4e797bfcba None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:52.651025 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-28dd0be2-9b65-4635-8734-1c4e797bfcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:52.739005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 958/1901] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:52.740012 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd0be2-9b65-4635-8734-1c4e797bfcba None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:52.741552 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-28dd0be2-9b65-4635-8734-1c4e797bfcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:52.756496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd0be2-9b65-4635-8734-1c4e797bfcba None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:52.762730 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:52.763563 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:52.763942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:52.764298 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] PATH_INFO: `/v3/users/284835da9a5e4035bc1b2d1011c22d63` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:52.765950 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=284835da9a5e4035bc1b2d1011c22d63)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:52.773346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:52.812146 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa2ea9356f44fc286f3bee2d28eb722'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:52.813185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa2ea9356f44fc286f3bee2d28eb722'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:52.875049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa2ea9356f44fc286f3bee2d28eb722'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:52.898379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 944/1902] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:52 2022] DELETE /identity/v3/users/6aa2ea9356f44fc286f3bee2d28eb722 => generated 0 bytes in 870 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:52.906924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:52.909419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:53.002625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '284835da9a5e4035bc1b2d1011c22d63'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:53.003569 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '284835da9a5e4035bc1b2d1011c22d63'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:53.037073 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7130d3d-f542-4d48-a44c-6a1894eedca9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:53.043389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829b998-da93-4ada-9b98-e0467bf8b290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:53.044431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829b998-da93-4ada-9b98-e0467bf8b290 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:53.044862 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829b998-da93-4ada-9b98-e0467bf8b290 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:53.045363 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829b998-da93-4ada-9b98-e0467bf8b290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:53.047343 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9829b998-da93-4ada-9b98-e0467bf8b290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:53.078943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '284835da9a5e4035bc1b2d1011c22d63'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:53.096610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:53.122787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '284835da9a5e4035bc1b2d1011c22d63'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:53.168212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '284835da9a5e4035bc1b2d1011c22d63'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:53.178221 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 959/1903] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:52 2022] DELETE /identity/v3/users/284835da9a5e4035bc1b2d1011c22d63 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:53.241862 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829b998-da93-4ada-9b98-e0467bf8b290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:53.243951 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829b998-da93-4ada-9b98-e0467bf8b290 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:53.284805 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 945/1904] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:54.208600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:54.211340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:54.326501 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:54.477423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82551ee-48ee-40db-9808-eac12b302da7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:54.487631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373ed07-8acd-4822-92a5-41ce0934fc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:54.489133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373ed07-8acd-4822-92a5-41ce0934fc4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:54.489873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373ed07-8acd-4822-92a5-41ce0934fc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:54.490729 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373ed07-8acd-4822-92a5-41ce0934fc4c admin admin] PATH_INFO: `/v3/projects/193804a0a9ab499d82bb78bdc58d9227` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:54.494074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373ed07-8acd-4822-92a5-41ce0934fc4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=193804a0a9ab499d82bb78bdc58d9227)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:54.498281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373ed07-8acd-4822-92a5-41ce0934fc4c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:54.662786 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f373ed07-8acd-4822-92a5-41ce0934fc4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '193804a0a9ab499d82bb78bdc58d9227', 'request_id': 'req-f373ed07-8acd-4822-92a5-41ce0934fc4c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:54.779775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f373ed07-8acd-4822-92a5-41ce0934fc4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '193804a0a9ab499d82bb78bdc58d9227', 'request_id': 'req-f373ed07-8acd-4822-92a5-41ce0934fc4c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:54.788429 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f373ed07-8acd-4822-92a5-41ce0934fc4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '193804a0a9ab499d82bb78bdc58d9227', 'request_id': 'req-f373ed07-8acd-4822-92a5-41ce0934fc4c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:54.814948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829b998-da93-4ada-9b98-e0467bf8b290 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:54.817174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9829b998-da93-4ada-9b98-e0467bf8b290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:54.835038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f373ed07-8acd-4822-92a5-41ce0934fc4c admin admin] The token cache is being invalidate because project 193804a0a9ab499d82bb78bdc58d9227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:54.835579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f373ed07-8acd-4822-92a5-41ce0934fc4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:54.838571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f373ed07-8acd-4822-92a5-41ce0934fc4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:54.840831 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829b998-da93-4ada-9b98-e0467bf8b290 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:54.843855 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 960/1905] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:54 2022] DELETE /identity/v3/projects/193804a0a9ab499d82bb78bdc58d9227 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:54.852890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522e64e-c04d-4923-b9fb-7f923abd167b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:54.853407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522e64e-c04d-4923-b9fb-7f923abd167b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:54.853972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522e64e-c04d-4923-b9fb-7f923abd167b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:54.920366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9522e64e-c04d-4923-b9fb-7f923abd167b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:54.924016 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9522e64e-c04d-4923-b9fb-7f923abd167b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:54.986652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c0286-2e2f-4b65-b586-71a96866fc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:54.989971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c0286-2e2f-4b65-b586-71a96866fc4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:54.990529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c0286-2e2f-4b65-b586-71a96866fc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:54.990948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c0286-2e2f-4b65-b586-71a96866fc4d admin admin] PATH_INFO: `/v3/projects/8e13fd935a2b4c01a2403f91c8eb818a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:54.995615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c0286-2e2f-4b65-b586-71a96866fc4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e13fd935a2b4c01a2403f91c8eb818a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:54.999436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c0286-2e2f-4b65-b586-71a96866fc4d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:55.202018 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 961/1906] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:55.217770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ab696-9e6f-4f92-938e-bfe0437b639b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:55.218965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ab696-9e6f-4f92-938e-bfe0437b639b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:55.218965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ab696-9e6f-4f92-938e-bfe0437b639b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:55.282196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d73ab696-9e6f-4f92-938e-bfe0437b639b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:55.285772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d73ab696-9e6f-4f92-938e-bfe0437b639b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:55.286011 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5c9c0286-2e2f-4b65-b586-71a96866fc4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e13fd935a2b4c01a2403f91c8eb818a', 'request_id': 'req-5c9c0286-2e2f-4b65-b586-71a96866fc4d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:55.366704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:55.407562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 962/1907] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:55.408835 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ab696-9e6f-4f92-938e-bfe0437b639b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:55.421116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ab696-9e6f-4f92-938e-bfe0437b639b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:55.423136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5c9c0286-2e2f-4b65-b586-71a96866fc4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e13fd935a2b4c01a2403f91c8eb818a', 'request_id': 'req-5c9c0286-2e2f-4b65-b586-71a96866fc4d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:55.427959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d077e39-0ce3-45b6-88c1-164a09c4ebb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:55.428655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d077e39-0ce3-45b6-88c1-164a09c4ebb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:55.428933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d077e39-0ce3-45b6-88c1-164a09c4ebb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:55.431558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d077e39-0ce3-45b6-88c1-164a09c4ebb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:55.431558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d077e39-0ce3-45b6-88c1-164a09c4ebb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:47:55.431558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d077e39-0ce3-45b6-88c1-164a09c4ebb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:55.433538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d077e39-0ce3-45b6-88c1-164a09c4ebb7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:55.455638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 963/1908] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:47:55.458497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d077e39-0ce3-45b6-88c1-164a09c4ebb7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:55.463208 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5c9c0286-2e2f-4b65-b586-71a96866fc4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e13fd935a2b4c01a2403f91c8eb818a', 'request_id': 'req-5c9c0286-2e2f-4b65-b586-71a96866fc4d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:55.498243 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d077e39-0ce3-45b6-88c1-164a09c4ebb7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:55.505442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5c9c0286-2e2f-4b65-b586-71a96866fc4d admin admin] The token cache is being invalidate because project 8e13fd935a2b4c01a2403f91c8eb818a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:55.505442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5c9c0286-2e2f-4b65-b586-71a96866fc4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:55.508633 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5c9c0286-2e2f-4b65-b586-71a96866fc4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:55.517718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 946/1909] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:54 2022] DELETE /identity/v3/projects/8e13fd935a2b4c01a2403f91c8eb818a => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:47:55.519651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75a34d-78f1-4499-b694-1e631eb79486 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:55.520158 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75a34d-78f1-4499-b694-1e631eb79486 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:55.520634 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75a34d-78f1-4499-b694-1e631eb79486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:55.523557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582dc68-eab6-488f-972c-8477f576165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:55.524421 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582dc68-eab6-488f-972c-8477f576165a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:55.524669 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582dc68-eab6-488f-972c-8477f576165a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:55.524941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582dc68-eab6-488f-972c-8477f576165a admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:55.525584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582dc68-eab6-488f-972c-8477f576165a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:55.527604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582dc68-eab6-488f-972c-8477f576165a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:55.568649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ec75a34d-78f1-4499-b694-1e631eb79486 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:55.570000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ec75a34d-78f1-4499-b694-1e631eb79486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:55.580060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 947/1910] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:55.590677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dac563-0054-4a5b-9dfa-3c653f6778a0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:55.590867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dac563-0054-4a5b-9dfa-3c653f6778a0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:55.591952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dac563-0054-4a5b-9dfa-3c653f6778a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:55.641059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-17dac563-0054-4a5b-9dfa-3c653f6778a0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:55.641059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-17dac563-0054-4a5b-9dfa-3c653f6778a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:55.653121 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 948/1911] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:47:55.654375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dac563-0054-4a5b-9dfa-3c653f6778a0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:55.656314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-17dac563-0054-4a5b-9dfa-3c653f6778a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:55.689873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dac563-0054-4a5b-9dfa-3c653f6778a0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:55.717222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:55.717222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:55.717222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:55.717222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] PATH_INFO: `/v3/projects/cbd52b8041114a18bb07601530a9a5a2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:55.718872 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbd52b8041114a18bb07601530a9a5a2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:55.719193 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 964/1912] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:55 2022] POST /identity/v3/projects => generated 387 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:55.720984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582dc68-eab6-488f-972c-8477f576165a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:55.721144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:55.830279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582dc68-eab6-488f-972c-8477f576165a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:55.851981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef7410-f3c6-4334-90d6-c85d626d2f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:55.852866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef7410-f3c6-4334-90d6-c85d626d2f49 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:55.853334 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef7410-f3c6-4334-90d6-c85d626d2f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:55.853757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef7410-f3c6-4334-90d6-c85d626d2f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:55.864760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef7410-f3c6-4334-90d6-c85d626d2f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:47:55.865078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef7410-f3c6-4334-90d6-c85d626d2f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:55.866998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef7410-f3c6-4334-90d6-c85d626d2f49 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:55.867309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:55.903099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 965/1913] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:47:55.904039 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef7410-f3c6-4334-90d6-c85d626d2f49 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:55.912862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e1ef7410-f3c6-4334-90d6-c85d626d2f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:55.928755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbd52b8041114a18bb07601530a9a5a2', 'request_id': 'req-75479973-c86c-4ff6-bf76-fdcd2e319691'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:55.944043 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef7410-f3c6-4334-90d6-c85d626d2f49 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:56.057515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c5128-ff78-4afe-aa78-e112061c0dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:56.059060 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c5128-ff78-4afe-aa78-e112061c0dbc admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:56.059060 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c5128-ff78-4afe-aa78-e112061c0dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:56.059060 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c5128-ff78-4afe-aa78-e112061c0dbc admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:56.060244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c5128-ff78-4afe-aa78-e112061c0dbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:56.064314 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c5128-ff78-4afe-aa78-e112061c0dbc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:56.065551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbd52b8041114a18bb07601530a9a5a2', 'request_id': 'req-75479973-c86c-4ff6-bf76-fdcd2e319691'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:56.072588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbd52b8041114a18bb07601530a9a5a2', 'request_id': 'req-75479973-c86c-4ff6-bf76-fdcd2e319691'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:56.088122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] The token cache is being invalidate because project cbd52b8041114a18bb07601530a9a5a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:56.088416 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:56.089197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:56.094135 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 949/1914] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:55 2022] DELETE /identity/v3/projects/cbd52b8041114a18bb07601530a9a5a2 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:56.095444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:56.097963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:56.117956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75479973-c86c-4ff6-bf76-fdcd2e319691 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:56.138111 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c76ea8-d24c-4075-85a5-c7a00afd540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:56.138777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c76ea8-d24c-4075-85a5-c7a00afd540d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:56.139054 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c76ea8-d24c-4075-85a5-c7a00afd540d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:56.139340 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c76ea8-d24c-4075-85a5-c7a00afd540d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:56.140182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c76ea8-d24c-4075-85a5-c7a00afd540d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:56.142355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c76ea8-d24c-4075-85a5-c7a00afd540d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:56.260087 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 966/1915] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:55 2022] POST /identity/v3/users => generated 294 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:56.260087 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c5128-ff78-4afe-aa78-e112061c0dbc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:56.261664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-767c5128-ff78-4afe-aa78-e112061c0dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:56.315618 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 950/1916] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:56 2022] POST /identity/v3/projects => generated 357 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:56.325323 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c76ea8-d24c-4075-85a5-c7a00afd540d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:56.334833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-22c76ea8-d24c-4075-85a5-c7a00afd540d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:56.357288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c5128-ff78-4afe-aa78-e112061c0dbc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:56.491604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:56.542793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:56.543481 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:56.547204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:56.547204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] PATH_INFO: `/v3/users/5136406fb78a477089e6538115cb58e9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:56.547204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5136406fb78a477089e6538115cb58e9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:56.547978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:56.616747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c76ea8-d24c-4075-85a5-c7a00afd540d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:56.622969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe6ecc-32e7-4841-b135-1c9903a36322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:56.623722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe6ecc-32e7-4841-b135-1c9903a36322 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:56.624079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe6ecc-32e7-4841-b135-1c9903a36322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:56.626780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe6ecc-32e7-4841-b135-1c9903a36322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:56.626780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe6ecc-32e7-4841-b135-1c9903a36322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:56.629816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe6ecc-32e7-4841-b135-1c9903a36322 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:56.644553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 951/1917] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:56.646434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe6ecc-32e7-4841-b135-1c9903a36322 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:56.659203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe6ecc-32e7-4841-b135-1c9903a36322 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:56.665744 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19daf8e-342f-4493-ae19-9162f6db2d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:56.667391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19daf8e-342f-4493-ae19-9162f6db2d86 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:56.667670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19daf8e-342f-4493-ae19-9162f6db2d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:56.668459 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19daf8e-342f-4493-ae19-9162f6db2d86 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:56.668963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19daf8e-342f-4493-ae19-9162f6db2d86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:56.671556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19daf8e-342f-4493-ae19-9162f6db2d86 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:56.812469 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 952/1918] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:56 2022] POST /identity/v3/users => generated 279 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:56.812469 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19daf8e-342f-4493-ae19-9162f6db2d86 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:56.839237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19daf8e-342f-4493-ae19-9162f6db2d86 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:56.848840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046e8d6-cde9-4422-90f2-bc475a0b6243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:56.850262 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046e8d6-cde9-4422-90f2-bc475a0b6243 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:56.850942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046e8d6-cde9-4422-90f2-bc475a0b6243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:56.852069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046e8d6-cde9-4422-90f2-bc475a0b6243 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:56.853491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046e8d6-cde9-4422-90f2-bc475a0b6243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:56.857251 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5046e8d6-cde9-4422-90f2-bc475a0b6243 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:56.878391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 953/1919] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:56.880121 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046e8d6-cde9-4422-90f2-bc475a0b6243 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:56.892660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5136406fb78a477089e6538115cb58e9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:56.894254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046e8d6-cde9-4422-90f2-bc475a0b6243 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:56.894483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5136406fb78a477089e6538115cb58e9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:56.900634 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3edd02-5693-4835-a52e-e74e0926175f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:56.902325 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3edd02-5693-4835-a52e-e74e0926175f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:56.902833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3edd02-5693-4835-a52e-e74e0926175f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:56.910638 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3edd02-5693-4835-a52e-e74e0926175f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:56.910638 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3edd02-5693-4835-a52e-e74e0926175f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:56.910638 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3edd02-5693-4835-a52e-e74e0926175f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:56.920164 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 954/1920] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:56.924575 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3edd02-5693-4835-a52e-e74e0926175f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:56.928100 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5d3edd02-5693-4835-a52e-e74e0926175f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:56.954005 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3edd02-5693-4835-a52e-e74e0926175f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:56.961252 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a73724-638b-49ab-bf2c-87127dec4fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:56.962374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a73724-638b-49ab-bf2c-87127dec4fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:56.962816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a73724-638b-49ab-bf2c-87127dec4fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:56.964055 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a73724-638b-49ab-bf2c-87127dec4fa1 admin admin] PATH_INFO: `/v3/projects/3f812d120f01455e9ac1c146bac80ec9/users/5bb447d4f3bf43a09ffa05ca6d216f22/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:56.985902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5136406fb78a477089e6538115cb58e9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:57.009232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a73724-638b-49ab-bf2c-87127dec4fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f812d120f01455e9ac1c146bac80ec9, user_id=5bb447d4f3bf43a09ffa05ca6d216f22, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:57.012681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a73724-638b-49ab-bf2c-87127dec4fa1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:57.015516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5136406fb78a477089e6538115cb58e9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:57.052202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5136406fb78a477089e6538115cb58e9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:57.067521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 967/1921] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:56 2022] DELETE /identity/v3/users/5136406fb78a477089e6538115cb58e9 => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:57.069228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:57.083150 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:57.129374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 955/1922] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:56 2022] PUT /identity/v3/projects/3f812d120f01455e9ac1c146bac80ec9/users/5bb447d4f3bf43a09ffa05ca6d216f22/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:57.151501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2001a-5796-4056-90fe-87ba4c995a5d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:57.151959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2001a-5796-4056-90fe-87ba4c995a5d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:57.153388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2001a-5796-4056-90fe-87ba4c995a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:57.212594 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-61d2001a-5796-4056-90fe-87ba4c995a5d None None] MFA Rules not processed for user `5bb447d4f3bf43a09ffa05ca6d216f22`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:57.215778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-61d2001a-5796-4056-90fe-87ba4c995a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:57.280803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0929fbc-469e-47b0-a4d9-7ddd1d229c85 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:57.286034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:57.442906 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 956/1923] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:47:57.443885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2001a-5796-4056-90fe-87ba4c995a5d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:57.455073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97459a9-a2b1-41c4-aafc-3f6043fe7045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:57.468952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97459a9-a2b1-41c4-aafc-3f6043fe7045 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:57.469485 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97459a9-a2b1-41c4-aafc-3f6043fe7045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:57.469952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97459a9-a2b1-41c4-aafc-3f6043fe7045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:57.470961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97459a9-a2b1-41c4-aafc-3f6043fe7045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:57.473035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97459a9-a2b1-41c4-aafc-3f6043fe7045 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:57.477926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:57.483657 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2001a-5796-4056-90fe-87ba4c995a5d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:57.501827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1d77b-b850-4fbc-aed9-a5b663de8c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:57.502228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 968/1924] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:57.502721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1d77b-b850-4fbc-aed9-a5b663de8c3a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:57.503104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1d77b-b850-4fbc-aed9-a5b663de8c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:57.504106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1d77b-b850-4fbc-aed9-a5b663de8c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:57.504238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97459a9-a2b1-41c4-aafc-3f6043fe7045 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:57.504332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1d77b-b850-4fbc-aed9-a5b663de8c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:57.507089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1d77b-b850-4fbc-aed9-a5b663de8c3a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:57.546614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97459a9-a2b1-41c4-aafc-3f6043fe7045 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:57.556499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9215f-9cc6-40f2-9b24-27b9175ec2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:57.556499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9215f-9cc6-40f2-9b24-27b9175ec2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:57.556782 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9215f-9cc6-40f2-9b24-27b9175ec2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:57.557059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9215f-9cc6-40f2-9b24-27b9175ec2dc admin admin] PATH_INFO: `/v3/projects/a7aaa023e2dd4ed991cedf3c4782d596/users/e745de2a7d904bf1889e7b7770cf6df1/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:57.577824 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9215f-9cc6-40f2-9b24-27b9175ec2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7aaa023e2dd4ed991cedf3c4782d596, user_id=e745de2a7d904bf1889e7b7770cf6df1, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:57.580826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9215f-9cc6-40f2-9b24-27b9175ec2dc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:57.677020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 957/1925] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:57 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:57.682702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1d77b-b850-4fbc-aed9-a5b663de8c3a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:57.706914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 969/1926] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:57 2022] PUT /identity/v3/projects/a7aaa023e2dd4ed991cedf3c4782d596/users/e745de2a7d904bf1889e7b7770cf6df1/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:57.733725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7af993-6a99-4f0d-919b-4135409b4755 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:57.733929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7af993-6a99-4f0d-919b-4135409b4755 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:57.734185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7af993-6a99-4f0d-919b-4135409b4755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:57.804347 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9b7af993-6a99-4f0d-919b-4135409b4755 None None] MFA Rules not processed for user `e745de2a7d904bf1889e7b7770cf6df1`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:57.806219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9b7af993-6a99-4f0d-919b-4135409b4755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:57.847090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1d77b-b850-4fbc-aed9-a5b663de8c3a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:57.945048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bf350-142b-4902-92f7-5b0726b9f190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:57.946632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bf350-142b-4902-92f7-5b0726b9f190 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:57.946632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bf350-142b-4902-92f7-5b0726b9f190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:57.947064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bf350-142b-4902-92f7-5b0726b9f190 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:57.947808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bf350-142b-4902-92f7-5b0726b9f190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:57.949804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bf350-142b-4902-92f7-5b0726b9f190 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:57.959214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 970/1927] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 11 08:47:57.960433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7af993-6a99-4f0d-919b-4135409b4755 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:57.962553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9b7af993-6a99-4f0d-919b-4135409b4755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:57.988522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7af993-6a99-4f0d-919b-4135409b4755 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:57.994568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:57.995228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:57.995509 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:57.995782 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] PATH_INFO: `/v3/projects/35541df7f55a467db29ece8f8b01c1e2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:57.997004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] RBAC: Authorizing `identity:delete_project(project_id=35541df7f55a467db29ece8f8b01c1e2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:57.999311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:58.085451 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 958/1928] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:57 2022] POST /identity/v3/users => generated 299 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:58.085451 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bf350-142b-4902-92f7-5b0726b9f190 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:58.088489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b17bf350-142b-4902-92f7-5b0726b9f190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:58.165687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35541df7f55a467db29ece8f8b01c1e2', 'request_id': 'req-70c2178d-2f03-42c9-acfe-307e0f79783c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:58.186780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:58.205765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bf350-142b-4902-92f7-5b0726b9f190 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:58.236519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35541df7f55a467db29ece8f8b01c1e2', 'request_id': 'req-70c2178d-2f03-42c9-acfe-307e0f79783c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:58.251898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35541df7f55a467db29ece8f8b01c1e2', 'request_id': 'req-70c2178d-2f03-42c9-acfe-307e0f79783c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:58.275712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01023e7-042b-409a-9308-b135025a21db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:58.276430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01023e7-042b-409a-9308-b135025a21db service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:58.276978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01023e7-042b-409a-9308-b135025a21db service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:58.277162 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] The token cache is being invalidate because project 35541df7f55a467db29ece8f8b01c1e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:47:58.277642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:58.278333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01023e7-042b-409a-9308-b135025a21db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:58.279334 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:58.280825 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 971/1929] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:47:57 2022] DELETE /identity/v3/projects/35541df7f55a467db29ece8f8b01c1e2 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:58.283653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c01023e7-042b-409a-9308-b135025a21db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:58.286859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:58.288305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:58.401943 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01023e7-042b-409a-9308-b135025a21db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:58.404966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01023e7-042b-409a-9308-b135025a21db service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:58.409449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:58.426039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 959/1930] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:58.435049 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2178d-2f03-42c9-acfe-307e0f79783c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:58.448288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d729b7-a4c6-4609-914b-6703b580bf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:58.448288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d729b7-a4c6-4609-914b-6703b580bf13 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:58.448288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d729b7-a4c6-4609-914b-6703b580bf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:58.448288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d729b7-a4c6-4609-914b-6703b580bf13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:58.448603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d729b7-a4c6-4609-914b-6703b580bf13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:58.448603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d729b7-a4c6-4609-914b-6703b580bf13 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:58.449870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573b4ff-abf5-4f3f-af37-f7da6215b7f7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:58.450798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573b4ff-abf5-4f3f-af37-f7da6215b7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:58.451104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573b4ff-abf5-4f3f-af37-f7da6215b7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:58.478724 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 972/1931] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:58.480386 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d729b7-a4c6-4609-914b-6703b580bf13 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:58.483506 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c2d729b7-a4c6-4609-914b-6703b580bf13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:58.509906 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7573b4ff-abf5-4f3f-af37-f7da6215b7f7 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:58.512250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7573b4ff-abf5-4f3f-af37-f7da6215b7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:58.513195 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d729b7-a4c6-4609-914b-6703b580bf13 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:58.531937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4f29e-2a0d-4857-bf20-5c16f4af55f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:58.533072 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4f29e-2a0d-4857-bf20-5c16f4af55f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:58.535695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4f29e-2a0d-4857-bf20-5c16f4af55f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:58.536236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4f29e-2a0d-4857-bf20-5c16f4af55f7 admin admin] PATH_INFO: `/v3/users/e745de2a7d904bf1889e7b7770cf6df1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:58.537083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 960/1932] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:58.538928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573b4ff-abf5-4f3f-af37-f7da6215b7f7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:58.543404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4f29e-2a0d-4857-bf20-5c16f4af55f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e745de2a7d904bf1889e7b7770cf6df1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:58.546215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4f29e-2a0d-4857-bf20-5c16f4af55f7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:58.566047 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573b4ff-abf5-4f3f-af37-f7da6215b7f7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:58.579534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9d06d-0e60-4358-9497-c2310ee767cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:58.580664 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9d06d-0e60-4358-9497-c2310ee767cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:58.581234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9d06d-0e60-4358-9497-c2310ee767cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:58.581699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9d06d-0e60-4358-9497-c2310ee767cd admin admin] PATH_INFO: `/v3/projects/ea145e4b563e4092824de0f316f68505/users/9884461cd95647ca93a2a65b991134d8/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:58.637794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9d06d-0e60-4358-9497-c2310ee767cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea145e4b563e4092824de0f316f68505, user_id=9884461cd95647ca93a2a65b991134d8, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:58.642348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9d06d-0e60-4358-9497-c2310ee767cd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:58.826966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 961/1933] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:58 2022] PUT /identity/v3/projects/ea145e4b563e4092824de0f316f68505/users/9884461cd95647ca93a2a65b991134d8/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:58.843851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143e1c1-ece7-4967-aae7-96b65b693de9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:58.843851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143e1c1-ece7-4967-aae7-96b65b693de9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:58.843851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143e1c1-ece7-4967-aae7-96b65b693de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:58.861284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9143e1c1-ece7-4967-aae7-96b65b693de9 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:58.862563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9143e1c1-ece7-4967-aae7-96b65b693de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:58.986572 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 962/1934] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:58.986572 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143e1c1-ece7-4967-aae7-96b65b693de9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:58.995414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-80f4f29e-2a0d-4857-bf20-5c16f4af55f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e745de2a7d904bf1889e7b7770cf6df1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:58.996134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-80f4f29e-2a0d-4857-bf20-5c16f4af55f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e745de2a7d904bf1889e7b7770cf6df1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:59.001518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143e1c1-ece7-4967-aae7-96b65b693de9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:59.009551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb8c24-a4be-4b3a-b409-53ec59dbc4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:59.010638 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb8c24-a4be-4b3a-b409-53ec59dbc4cc admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:59.011082 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb8c24-a4be-4b3a-b409-53ec59dbc4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:59.011576 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb8c24-a4be-4b3a-b409-53ec59dbc4cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:59.012785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb8c24-a4be-4b3a-b409-53ec59dbc4cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:59.015913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb8c24-a4be-4b3a-b409-53ec59dbc4cc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:59.036646 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 963/1935] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:59.038130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb8c24-a4be-4b3a-b409-53ec59dbc4cc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:59.041559 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7deb8c24-a4be-4b3a-b409-53ec59dbc4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:59.064926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:59.073897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb8c24-a4be-4b3a-b409-53ec59dbc4cc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:59.082211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e38325-d430-4bb1-93c3-0321bbba19eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:59.083266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e38325-d430-4bb1-93c3-0321bbba19eb admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:59.083916 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e38325-d430-4bb1-93c3-0321bbba19eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:59.084408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e38325-d430-4bb1-93c3-0321bbba19eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:59.085623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e38325-d430-4bb1-93c3-0321bbba19eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:47:59.086730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e38325-d430-4bb1-93c3-0321bbba19eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:59.091830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e38325-d430-4bb1-93c3-0321bbba19eb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:59.106802 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 964/1936] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:47:59.107919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e38325-d430-4bb1-93c3-0321bbba19eb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:59.112964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-49e38325-d430-4bb1-93c3-0321bbba19eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:59.128715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e38325-d430-4bb1-93c3-0321bbba19eb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:59.134983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8588b-6f17-4ec2-a395-2d1fb5cc5d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:59.135829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8588b-6f17-4ec2-a395-2d1fb5cc5d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:59.136124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8588b-6f17-4ec2-a395-2d1fb5cc5d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:59.136431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8588b-6f17-4ec2-a395-2d1fb5cc5d9d admin admin] PATH_INFO: `/v3/projects/ea145e4b563e4092824de0f316f68505/users/9884461cd95647ca93a2a65b991134d8/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:59.139158 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c8588b-6f17-4ec2-a395-2d1fb5cc5d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea145e4b563e4092824de0f316f68505, user_id=9884461cd95647ca93a2a65b991134d8, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:59.143909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c8588b-6f17-4ec2-a395-2d1fb5cc5d9d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:59.157978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 965/1937] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:59 2022] PUT /identity/v3/projects/ea145e4b563e4092824de0f316f68505/users/9884461cd95647ca93a2a65b991134d8/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:59.166132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-80f4f29e-2a0d-4857-bf20-5c16f4af55f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e745de2a7d904bf1889e7b7770cf6df1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:59.169195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8588b-6f17-4ec2-a395-2d1fb5cc5d9d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:59.177840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-80f4f29e-2a0d-4857-bf20-5c16f4af55f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e745de2a7d904bf1889e7b7770cf6df1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:59.203693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-80f4f29e-2a0d-4857-bf20-5c16f4af55f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e745de2a7d904bf1889e7b7770cf6df1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:47:59.216627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 973/1938] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:47:58 2022] DELETE /identity/v3/users/e745de2a7d904bf1889e7b7770cf6df1 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:47:59.231206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dc199-1b1a-46a8-877b-b72cb6ff0c07 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:59.231934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dc199-1b1a-46a8-877b-b72cb6ff0c07 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:59.232745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dc199-1b1a-46a8-877b-b72cb6ff0c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:59.279704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8588b-6f17-4ec2-a395-2d1fb5cc5d9d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:59.282432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-367dc199-1b1a-46a8-877b-b72cb6ff0c07 None None] MFA Rules not processed for user `9884461cd95647ca93a2a65b991134d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:59.283762 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-367dc199-1b1a-46a8-877b-b72cb6ff0c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:59.362985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7325a-76d4-4f8f-8093-76ba90101cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:59.364102 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7325a-76d4-4f8f-8093-76ba90101cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:59.364599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7325a-76d4-4f8f-8093-76ba90101cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:59.365140 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7325a-76d4-4f8f-8093-76ba90101cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:59.365906 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7325a-76d4-4f8f-8093-76ba90101cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:59.368572 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7325a-76d4-4f8f-8093-76ba90101cb5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:59.389565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 974/1939] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:59.392472 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e7f7e-bb35-4934-ab39-f614a23e0db8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:59.393023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e7f7e-bb35-4934-ab39-f614a23e0db8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:59.393500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e7f7e-bb35-4934-ab39-f614a23e0db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:59.422339 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a50e7f7e-bb35-4934-ab39-f614a23e0db8 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:47:59.423774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a50e7f7e-bb35-4934-ab39-f614a23e0db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:59.431879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 975/1940] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:47:59.432872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e7f7e-bb35-4934-ab39-f614a23e0db8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:59.434379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a50e7f7e-bb35-4934-ab39-f614a23e0db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:59.489888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:59.498575 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 966/1941] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:47:59 2022] POST /identity/v3/projects => generated 373 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:59.503796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7325a-76d4-4f8f-8093-76ba90101cb5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:59.506417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-66f7325a-76d4-4f8f-8093-76ba90101cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:47:59.533077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e7f7e-bb35-4934-ab39-f614a23e0db8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:59.595493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7325a-76d4-4f8f-8093-76ba90101cb5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:59.599636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264d131-9c5c-4557-99da-4b16547297a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:59.600452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264d131-9c5c-4557-99da-4b16547297a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:59.600917 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264d131-9c5c-4557-99da-4b16547297a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:59.601159 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264d131-9c5c-4557-99da-4b16547297a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:59.601620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1a527-2c0b-4f98-aca8-e07dc31a0ff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:59.602430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1a527-2c0b-4f98-aca8-e07dc31a0ff6 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:59.602706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1a527-2c0b-4f98-aca8-e07dc31a0ff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:59.602969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1a527-2c0b-4f98-aca8-e07dc31a0ff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:59.631047 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:47:59.683433 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264d131-9c5c-4557-99da-4b16547297a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:59.685143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264d131-9c5c-4557-99da-4b16547297a3 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:59.685687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1a527-2c0b-4f98-aca8-e07dc31a0ff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:59.690859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1a527-2c0b-4f98-aca8-e07dc31a0ff6 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:59.706890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 967/1942] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:59.708850 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264d131-9c5c-4557-99da-4b16547297a3 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:59.709381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 976/1943] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:47:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:47:59.728294 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264d131-9c5c-4557-99da-4b16547297a3 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:59.736366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d031cb-9803-47bb-bda6-29a06960e8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:59.743510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d031cb-9803-47bb-bda6-29a06960e8bb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:59.743510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d031cb-9803-47bb-bda6-29a06960e8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:59.743510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d031cb-9803-47bb-bda6-29a06960e8bb admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:59.743510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d031cb-9803-47bb-bda6-29a06960e8bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:59.743510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d031cb-9803-47bb-bda6-29a06960e8bb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:59.888055 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 968/1944] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:47:59 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:47:59.892470 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1a527-2c0b-4f98-aca8-e07dc31a0ff6 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:59.911045 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1a527-2c0b-4f98-aca8-e07dc31a0ff6 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:59.920362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b06c50-68af-40df-9055-303101c97a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:59.921377 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b06c50-68af-40df-9055-303101c97a5d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:59.921728 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b06c50-68af-40df-9055-303101c97a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:59.922076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b06c50-68af-40df-9055-303101c97a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:59.923545 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b06c50-68af-40df-9055-303101c97a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:59.924789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b06c50-68af-40df-9055-303101c97a5d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:59.939235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 977/1945] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:59.943211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d031cb-9803-47bb-bda6-29a06960e8bb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:47:59.956482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d031cb-9803-47bb-bda6-29a06960e8bb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:47:59.965635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf067e8-e34a-40f7-9570-5a8c3fcd0432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:47:59.967661 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf067e8-e34a-40f7-9570-5a8c3fcd0432 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:47:59.967946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf067e8-e34a-40f7-9570-5a8c3fcd0432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:47:59.968218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf067e8-e34a-40f7-9570-5a8c3fcd0432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:47:59.968885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf067e8-e34a-40f7-9570-5a8c3fcd0432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:47:59.970735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf067e8-e34a-40f7-9570-5a8c3fcd0432 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:47:59.986028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 969/1946] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:47:59.990529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b06c50-68af-40df-9055-303101c97a5d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:00.029519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b06c50-68af-40df-9055-303101c97a5d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:00.035350 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7053a-78b9-4049-85cd-05c4d0495522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:00.035836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7053a-78b9-4049-85cd-05c4d0495522 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:00.036258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7053a-78b9-4049-85cd-05c4d0495522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:00.036543 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7053a-78b9-4049-85cd-05c4d0495522 admin admin] PATH_INFO: `/v3/projects/d22f00fe9e08438a81b9a3fa2bd03a93/users/b7bbf2a792da49bcab61f673ced3c779/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:00.070201 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7053a-78b9-4049-85cd-05c4d0495522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22f00fe9e08438a81b9a3fa2bd03a93, user_id=b7bbf2a792da49bcab61f673ced3c779, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:00.076855 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7053a-78b9-4049-85cd-05c4d0495522 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:00.192986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 978/1947] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:47:59 2022] PUT /identity/v3/projects/d22f00fe9e08438a81b9a3fa2bd03a93/users/b7bbf2a792da49bcab61f673ced3c779/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:00.196726 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceafb90-26cf-440f-808f-cb479b89b55e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:00.199780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceafb90-26cf-440f-808f-cb479b89b55e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:00.204143 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceafb90-26cf-440f-808f-cb479b89b55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:00.290128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9ceafb90-26cf-440f-808f-cb479b89b55e None None] MFA Rules not processed for user `b7bbf2a792da49bcab61f673ced3c779`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:00.292516 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9ceafb90-26cf-440f-808f-cb479b89b55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:00.411650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 970/1948] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:00.419412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7053a-78b9-4049-85cd-05c4d0495522 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:00.500862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7053a-78b9-4049-85cd-05c4d0495522 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:00.505899 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe56285-6671-430c-8e1c-e1effcb302b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:00.506555 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe56285-6671-430c-8e1c-e1effcb302b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:00.506827 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe56285-6671-430c-8e1c-e1effcb302b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:00.507090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe56285-6671-430c-8e1c-e1effcb302b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:00.507743 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe56285-6671-430c-8e1c-e1effcb302b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:00.509742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe56285-6671-430c-8e1c-e1effcb302b6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:00.520358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:00.603810 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceafb90-26cf-440f-808f-cb479b89b55e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:00.606295 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9ceafb90-26cf-440f-808f-cb479b89b55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:00.624510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceafb90-26cf-440f-808f-cb479b89b55e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:00.629518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873baa6e-d78a-49cf-bc5e-fd966f509961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:00.630299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873baa6e-d78a-49cf-bc5e-fd966f509961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:00.630684 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873baa6e-d78a-49cf-bc5e-fd966f509961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:00.631080 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873baa6e-d78a-49cf-bc5e-fd966f509961 admin admin] PATH_INFO: `/v3/projects/a7aaa023e2dd4ed991cedf3c4782d596` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:00.632430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873baa6e-d78a-49cf-bc5e-fd966f509961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7aaa023e2dd4ed991cedf3c4782d596)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:00.634623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873baa6e-d78a-49cf-bc5e-fd966f509961 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:00.652196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 979/1949] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:00 2022] POST /identity/v3/projects => generated 373 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:48:00.658290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe56285-6671-430c-8e1c-e1effcb302b6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:00.779233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe56285-6671-430c-8e1c-e1effcb302b6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:00.785230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159aabe-828b-48c2-98c6-bf529d8dd4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:00.786173 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159aabe-828b-48c2-98c6-bf529d8dd4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:00.786513 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159aabe-828b-48c2-98c6-bf529d8dd4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:00.786852 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159aabe-828b-48c2-98c6-bf529d8dd4b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:00.787583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159aabe-828b-48c2-98c6-bf529d8dd4b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:00.790444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159aabe-828b-48c2-98c6-bf529d8dd4b0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:00.835134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-873baa6e-d78a-49cf-bc5e-fd966f509961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7aaa023e2dd4ed991cedf3c4782d596', 'request_id': 'req-873baa6e-d78a-49cf-bc5e-fd966f509961'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:00.924452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 980/1950] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:00 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:48:00.930567 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159aabe-828b-48c2-98c6-bf529d8dd4b0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:00.934335 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-873baa6e-d78a-49cf-bc5e-fd966f509961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7aaa023e2dd4ed991cedf3c4782d596', 'request_id': 'req-873baa6e-d78a-49cf-bc5e-fd966f509961'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:00.946377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-873baa6e-d78a-49cf-bc5e-fd966f509961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7aaa023e2dd4ed991cedf3c4782d596', 'request_id': 'req-873baa6e-d78a-49cf-bc5e-fd966f509961'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:00.948450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159aabe-828b-48c2-98c6-bf529d8dd4b0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:00.962559 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-873baa6e-d78a-49cf-bc5e-fd966f509961 admin admin] The token cache is being invalidate because project a7aaa023e2dd4ed991cedf3c4782d596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:00.962858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-873baa6e-d78a-49cf-bc5e-fd966f509961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:00.963628 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-873baa6e-d78a-49cf-bc5e-fd966f509961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:00.965577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 971/1951] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:00 2022] DELETE /identity/v3/projects/a7aaa023e2dd4ed991cedf3c4782d596 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:00.970174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a455c-0186-4c58-b8b5-d8eec6814cbc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:00.970471 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a455c-0186-4c58-b8b5-d8eec6814cbc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:00.970742 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a455c-0186-4c58-b8b5-d8eec6814cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:00.981329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:00.996126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dd1a455c-0186-4c58-b8b5-d8eec6814cbc None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:00.997488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dd1a455c-0186-4c58-b8b5-d8eec6814cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:01.070867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d35dfd-d8e8-437b-84ce-7ac01884ea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:01.072163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d35dfd-d8e8-437b-84ce-7ac01884ea59 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:01.072698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d35dfd-d8e8-437b-84ce-7ac01884ea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:01.073284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d35dfd-d8e8-437b-84ce-7ac01884ea59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:01.074881 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d35dfd-d8e8-437b-84ce-7ac01884ea59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:01.078734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d35dfd-d8e8-437b-84ce-7ac01884ea59 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:01.089076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 972/1952] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:01.099744 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce70b22-d140-4702-85ec-b1e6e7cda438 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:01.100196 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce70b22-d140-4702-85ec-b1e6e7cda438 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:01.100630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce70b22-d140-4702-85ec-b1e6e7cda438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:01.102586 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 981/1953] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:01.109583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d35dfd-d8e8-437b-84ce-7ac01884ea59 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:01.114138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c1d35dfd-d8e8-437b-84ce-7ac01884ea59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:01.144004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d35dfd-d8e8-437b-84ce-7ac01884ea59 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:01.153213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd045e9-1cce-434f-bd15-029f054c2ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:01.154398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd045e9-1cce-434f-bd15-029f054c2ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:01.154871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd045e9-1cce-434f-bd15-029f054c2ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:01.155329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd045e9-1cce-434f-bd15-029f054c2ab8 admin admin] PATH_INFO: `/v3/projects/6517311247dd4c8784c5ac19b6c5c072/users/d57a79d411e047c18d2ae40a69acb33c/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:01.156607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7ce70b22-d140-4702-85ec-b1e6e7cda438 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:01.159180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7ce70b22-d140-4702-85ec-b1e6e7cda438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:01.182325 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 973/1954] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:01.182325 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce70b22-d140-4702-85ec-b1e6e7cda438 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:01.186990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd045e9-1cce-434f-bd15-029f054c2ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6517311247dd4c8784c5ac19b6c5c072, user_id=d57a79d411e047c18d2ae40a69acb33c, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:01.190281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd045e9-1cce-434f-bd15-029f054c2ab8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:01.195022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce70b22-d140-4702-85ec-b1e6e7cda438 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:01.201336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ace23-3c79-4af2-b011-85f13488b98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:01.206855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ace23-3c79-4af2-b011-85f13488b98e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:01.206855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ace23-3c79-4af2-b011-85f13488b98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:01.206855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ace23-3c79-4af2-b011-85f13488b98e admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:01.206855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ace23-3c79-4af2-b011-85f13488b98e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:48:01.206855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ace23-3c79-4af2-b011-85f13488b98e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:01.206855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ace23-3c79-4af2-b011-85f13488b98e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:01.230183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 974/1955] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:48:01.240529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ace23-3c79-4af2-b011-85f13488b98e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:01.257805 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ace23-3c79-4af2-b011-85f13488b98e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:01.269361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a287c-1e2a-4b15-a795-a4a20f0ec620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:01.270747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a287c-1e2a-4b15-a795-a4a20f0ec620 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:01.271935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a287c-1e2a-4b15-a795-a4a20f0ec620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:01.272461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a287c-1e2a-4b15-a795-a4a20f0ec620 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:01.273219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a287c-1e2a-4b15-a795-a4a20f0ec620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:01.276118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a287c-1e2a-4b15-a795-a4a20f0ec620 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:01.323498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 982/1956] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:01 2022] PUT /identity/v3/projects/6517311247dd4c8784c5ac19b6c5c072/users/d57a79d411e047c18d2ae40a69acb33c/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:01.328701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd045e9-1cce-434f-bd15-029f054c2ab8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:01.410499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:01.423737 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd045e9-1cce-434f-bd15-029f054c2ab8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:01.430453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b2b12-2728-44b9-9d2b-be547a75f0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:01.431087 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b2b12-2728-44b9-9d2b-be547a75f0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:01.431364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b2b12-2728-44b9-9d2b-be547a75f0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:01.431633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b2b12-2728-44b9-9d2b-be547a75f0d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:01.433054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b2b12-2728-44b9-9d2b-be547a75f0d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:01.435384 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b2b12-2728-44b9-9d2b-be547a75f0d6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:01.445693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 975/1957] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:01 2022] POST /identity/v3/projects => generated 373 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:48:01.451605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 983/1958] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:01.451872 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a287c-1e2a-4b15-a795-a4a20f0ec620 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:01.452847 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b2b12-2728-44b9-9d2b-be547a75f0d6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:01.465382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b2b12-2728-44b9-9d2b-be547a75f0d6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:01.541842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a287c-1e2a-4b15-a795-a4a20f0ec620 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:01.546723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4c6df-8a25-4193-b314-597a2ca055da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:01.547486 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4c6df-8a25-4193-b314-597a2ca055da admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:01.547948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4c6df-8a25-4193-b314-597a2ca055da admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:01.548189 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4c6df-8a25-4193-b314-597a2ca055da admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:01.548881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d4c6df-8a25-4193-b314-597a2ca055da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:01.550975 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d4c6df-8a25-4193-b314-597a2ca055da admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:01.560495 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63778fec-ef53-4826-85a5-3cb972e726ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:01.561477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63778fec-ef53-4826-85a5-3cb972e726ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:01.561592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63778fec-ef53-4826-85a5-3cb972e726ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:01.561951 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63778fec-ef53-4826-85a5-3cb972e726ae admin admin] PATH_INFO: `/v3/projects/6517311247dd4c8784c5ac19b6c5c072/users/d57a79d411e047c18d2ae40a69acb33c/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:01.564500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63778fec-ef53-4826-85a5-3cb972e726ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=6517311247dd4c8784c5ac19b6c5c072, user_id=d57a79d411e047c18d2ae40a69acb33c, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:01.568188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63778fec-ef53-4826-85a5-3cb972e726ae admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:01.585450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 984/1959] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:01 2022] PUT /identity/v3/projects/6517311247dd4c8784c5ac19b6c5c072/users/d57a79d411e047c18d2ae40a69acb33c/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:01.598057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068482fb-6bff-48d0-88a7-dbb75e4422a5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:01.598573 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068482fb-6bff-48d0-88a7-dbb75e4422a5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:01.598909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068482fb-6bff-48d0-88a7-dbb75e4422a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:01.638844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-068482fb-6bff-48d0-88a7-dbb75e4422a5 None None] MFA Rules not processed for user `d57a79d411e047c18d2ae40a69acb33c`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:01.640380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-068482fb-6bff-48d0-88a7-dbb75e4422a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:01.690176 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 976/1960] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:01 2022] POST /identity/v3/users => generated 287 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:48:01.694343 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4c6df-8a25-4193-b314-597a2ca055da admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:01.708417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4c6df-8a25-4193-b314-597a2ca055da admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:01.729538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 985/1961] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:01.759789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068482fb-6bff-48d0-88a7-dbb75e4422a5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:01.761767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-068482fb-6bff-48d0-88a7-dbb75e4422a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:01.781422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:01.810184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695ed17-38f9-464c-9252-4d2a3bad6988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:01.810184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695ed17-38f9-464c-9252-4d2a3bad6988 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:01.810184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695ed17-38f9-464c-9252-4d2a3bad6988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:01.810184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695ed17-38f9-464c-9252-4d2a3bad6988 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:01.816030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695ed17-38f9-464c-9252-4d2a3bad6988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:01.816030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695ed17-38f9-464c-9252-4d2a3bad6988 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:01.816030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 977/1962] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:01.819831 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695ed17-38f9-464c-9252-4d2a3bad6988 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:01.835083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695ed17-38f9-464c-9252-4d2a3bad6988 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:01.844692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc062d8-ae8b-4602-a15e-7f8b544ec084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:01.844692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc062d8-ae8b-4602-a15e-7f8b544ec084 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:01.844692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc062d8-ae8b-4602-a15e-7f8b544ec084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:01.844692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc062d8-ae8b-4602-a15e-7f8b544ec084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:01.846212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc062d8-ae8b-4602-a15e-7f8b544ec084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:01.847505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc062d8-ae8b-4602-a15e-7f8b544ec084 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:01.863349 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068482fb-6bff-48d0-88a7-dbb75e4422a5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:01.863604 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 978/1963] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:01.867712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc062d8-ae8b-4602-a15e-7f8b544ec084 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:01.874039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9fc062d8-ae8b-4602-a15e-7f8b544ec084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:01.886023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d75e73-8e73-4c61-a310-695f038643d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:01.890406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d75e73-8e73-4c61-a310-695f038643d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:01.892279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d75e73-8e73-4c61-a310-695f038643d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:01.892814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d75e73-8e73-4c61-a310-695f038643d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:01.896143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-85d75e73-8e73-4c61-a310-695f038643d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:01.899142 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc062d8-ae8b-4602-a15e-7f8b544ec084 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:01.908808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a8234-bcf9-46f2-b55c-13f574d6f36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:01.910300 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a8234-bcf9-46f2-b55c-13f574d6f36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:01.911532 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a8234-bcf9-46f2-b55c-13f574d6f36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:01.911879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a8234-bcf9-46f2-b55c-13f574d6f36f admin admin] PATH_INFO: `/v3/users/5bb447d4f3bf43a09ffa05ca6d216f22` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:01.915672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a8234-bcf9-46f2-b55c-13f574d6f36f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bb447d4f3bf43a09ffa05ca6d216f22)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:01.920655 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a8234-bcf9-46f2-b55c-13f574d6f36f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:01.924668 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:01.983040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d75e73-8e73-4c61-a310-695f038643d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:01.986073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d75e73-8e73-4c61-a310-695f038643d9 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:02.000601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 986/1964] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:48:02.001470 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d75e73-8e73-4c61-a310-695f038643d9 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:02.014273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d75e73-8e73-4c61-a310-695f038643d9 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:02.020854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73a49a-370b-432d-8802-f1dfac4b2370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:02.021807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73a49a-370b-432d-8802-f1dfac4b2370 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:02.022209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73a49a-370b-432d-8802-f1dfac4b2370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:02.022690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73a49a-370b-432d-8802-f1dfac4b2370 admin admin] PATH_INFO: `/v3/projects/ead98e6e89d840a0b99273f28429b4ba/users/0b198ff6ed104068b38b993054617f9a/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:02.042998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73a49a-370b-432d-8802-f1dfac4b2370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ead98e6e89d840a0b99273f28429b4ba, user_id=0b198ff6ed104068b38b993054617f9a, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:02.046531 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73a49a-370b-432d-8802-f1dfac4b2370 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:02.140602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 987/1965] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:02 2022] PUT /identity/v3/projects/ead98e6e89d840a0b99273f28429b4ba/users/0b198ff6ed104068b38b993054617f9a/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:02.148049 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0a04b-d80d-4126-8e60-2f9bb8ceabd7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:02.149024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0a04b-d80d-4126-8e60-2f9bb8ceabd7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:02.149479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0a04b-d80d-4126-8e60-2f9bb8ceabd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:02.182547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ccb0a04b-d80d-4126-8e60-2f9bb8ceabd7 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:02.184327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ccb0a04b-d80d-4126-8e60-2f9bb8ceabd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:02.187899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-722a8234-bcf9-46f2-b55c-13f574d6f36f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bb447d4f3bf43a09ffa05ca6d216f22'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:02.207004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-722a8234-bcf9-46f2-b55c-13f574d6f36f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bb447d4f3bf43a09ffa05ca6d216f22'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:02.261451 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 988/1966] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:02.271602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0a04b-d80d-4126-8e60-2f9bb8ceabd7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:02.274165 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ccb0a04b-d80d-4126-8e60-2f9bb8ceabd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:02.283113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-722a8234-bcf9-46f2-b55c-13f574d6f36f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bb447d4f3bf43a09ffa05ca6d216f22'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:02.284345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-722a8234-bcf9-46f2-b55c-13f574d6f36f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bb447d4f3bf43a09ffa05ca6d216f22'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:02.299256 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-722a8234-bcf9-46f2-b55c-13f574d6f36f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bb447d4f3bf43a09ffa05ca6d216f22'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:02.310924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 979/1967] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:01 2022] DELETE /identity/v3/users/5bb447d4f3bf43a09ffa05ca6d216f22 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:02.318223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4a2b1-bf73-4310-8673-67e60f18bf69 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:02.318716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4a2b1-bf73-4310-8673-67e60f18bf69 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:02.319195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4a2b1-bf73-4310-8673-67e60f18bf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:02.341398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-04f4a2b1-bf73-4310-8673-67e60f18bf69 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:02.342694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-04f4a2b1-bf73-4310-8673-67e60f18bf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:02.347015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0a04b-d80d-4126-8e60-2f9bb8ceabd7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:02.413630 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185e6f4-7361-4213-a3d8-a4e543abca5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:02.415245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185e6f4-7361-4213-a3d8-a4e543abca5d service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:02.415589 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185e6f4-7361-4213-a3d8-a4e543abca5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:02.416038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185e6f4-7361-4213-a3d8-a4e543abca5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:02.419166 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8185e6f4-7361-4213-a3d8-a4e543abca5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:02.436655 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 980/1968] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:02.452328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4a2b1-bf73-4310-8673-67e60f18bf69 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:02.453769 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-04f4a2b1-bf73-4310-8673-67e60f18bf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:02.467102 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4a2b1-bf73-4310-8673-67e60f18bf69 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:02.472699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee498a-0a82-4ac7-a9ce-78b458c04261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:02.473324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee498a-0a82-4ac7-a9ce-78b458c04261 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:02.484511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee498a-0a82-4ac7-a9ce-78b458c04261 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:02.484511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee498a-0a82-4ac7-a9ce-78b458c04261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:02.486255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee498a-0a82-4ac7-a9ce-78b458c04261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:02.489076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee498a-0a82-4ac7-a9ce-78b458c04261 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:02.504058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 981/1969] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:48:02.504285 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185e6f4-7361-4213-a3d8-a4e543abca5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:02.505416 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185e6f4-7361-4213-a3d8-a4e543abca5d service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:02.517119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 989/1970] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:48:02.567869 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:03.132906 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee498a-0a82-4ac7-a9ce-78b458c04261 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:03.140704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d6ee498a-0a82-4ac7-a9ce-78b458c04261 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:03.168616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee498a-0a82-4ac7-a9ce-78b458c04261 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:03.181238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac29e9-3604-4fc5-bc47-812aaea3bc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:03.187270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac29e9-3604-4fc5-bc47-812aaea3bc06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:03.187270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac29e9-3604-4fc5-bc47-812aaea3bc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:03.187270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac29e9-3604-4fc5-bc47-812aaea3bc06 admin admin] PATH_INFO: `/v3/projects/3f812d120f01455e9ac1c146bac80ec9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:03.187270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac29e9-3604-4fc5-bc47-812aaea3bc06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f812d120f01455e9ac1c146bac80ec9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:03.187270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac29e9-3604-4fc5-bc47-812aaea3bc06 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:03.334775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:03.378517 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-77ac29e9-3604-4fc5-bc47-812aaea3bc06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f812d120f01455e9ac1c146bac80ec9', 'request_id': 'req-77ac29e9-3604-4fc5-bc47-812aaea3bc06'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:03.518522 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-77ac29e9-3604-4fc5-bc47-812aaea3bc06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f812d120f01455e9ac1c146bac80ec9', 'request_id': 'req-77ac29e9-3604-4fc5-bc47-812aaea3bc06'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:03.530211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-77ac29e9-3604-4fc5-bc47-812aaea3bc06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f812d120f01455e9ac1c146bac80ec9', 'request_id': 'req-77ac29e9-3604-4fc5-bc47-812aaea3bc06'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:03.548995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-77ac29e9-3604-4fc5-bc47-812aaea3bc06 admin admin] The token cache is being invalidate because project 3f812d120f01455e9ac1c146bac80ec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:03.550825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-77ac29e9-3604-4fc5-bc47-812aaea3bc06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:03.551116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-77ac29e9-3604-4fc5-bc47-812aaea3bc06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:03.552762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 982/1971] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:03 2022] DELETE /identity/v3/projects/3f812d120f01455e9ac1c146bac80ec9 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:03.558068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185e6f4-7361-4213-a3d8-a4e543abca5d service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:03.571304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185e6f4-7361-4213-a3d8-a4e543abca5d service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:03.641897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:03.642586 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:03.642896 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:03.643322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] PATH_INFO: `/v3/users/9884461cd95647ca93a2a65b991134d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:03.645944 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9884461cd95647ca93a2a65b991134d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:03.650462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:03.893806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9884461cd95647ca93a2a65b991134d8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:03.894525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9884461cd95647ca93a2a65b991134d8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:04.017501 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9884461cd95647ca93a2a65b991134d8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:04.028042 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9884461cd95647ca93a2a65b991134d8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:04.056975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9884461cd95647ca93a2a65b991134d8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:04.069805 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 990/1972] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:03 2022] DELETE /identity/v3/users/9884461cd95647ca93a2a65b991134d8 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:04.194859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc81741-ddaf-4c54-a31a-3fed12dea18c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:04.195654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc81741-ddaf-4c54-a31a-3fed12dea18c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:04.196149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc81741-ddaf-4c54-a31a-3fed12dea18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:04.247356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fcc81741-ddaf-4c54-a31a-3fed12dea18c None None] MFA Rules not processed for user `0b198ff6ed104068b38b993054617f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:04.248838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fcc81741-ddaf-4c54-a31a-3fed12dea18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:04.414880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 983/1973] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:04.423483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:04.426536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:04.469832 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:04.522424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f92da-7b5f-4229-bb1f-d535cfa2b210 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:04.533258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf0e1e-d9ec-443b-99ca-b7624c9c556e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:04.536065 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf0e1e-d9ec-443b-99ca-b7624c9c556e service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:04.536977 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf0e1e-d9ec-443b-99ca-b7624c9c556e service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:04.537491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf0e1e-d9ec-443b-99ca-b7624c9c556e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:04.560927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf0e1e-d9ec-443b-99ca-b7624c9c556e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:04.565099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf0e1e-d9ec-443b-99ca-b7624c9c556e service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:04.589689 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 991/1974] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:48:04.848380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc81741-ddaf-4c54-a31a-3fed12dea18c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:04.850593 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fcc81741-ddaf-4c54-a31a-3fed12dea18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:04.941196 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc81741-ddaf-4c54-a31a-3fed12dea18c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:04.947115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ef867-38a6-4504-a4c4-34a5280bc342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:04.947756 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ef867-38a6-4504-a4c4-34a5280bc342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:04.948024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ef867-38a6-4504-a4c4-34a5280bc342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:04.948291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ef867-38a6-4504-a4c4-34a5280bc342 admin admin] PATH_INFO: `/v3/projects/ea145e4b563e4092824de0f316f68505` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:04.949449 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ef867-38a6-4504-a4c4-34a5280bc342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea145e4b563e4092824de0f316f68505)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:04.951408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ef867-38a6-4504-a4c4-34a5280bc342 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:05.139404 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8a3ef867-38a6-4504-a4c4-34a5280bc342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea145e4b563e4092824de0f316f68505', 'request_id': 'req-8a3ef867-38a6-4504-a4c4-34a5280bc342'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:05.142134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:05.253531 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8a3ef867-38a6-4504-a4c4-34a5280bc342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea145e4b563e4092824de0f316f68505', 'request_id': 'req-8a3ef867-38a6-4504-a4c4-34a5280bc342'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:05.261722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8a3ef867-38a6-4504-a4c4-34a5280bc342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea145e4b563e4092824de0f316f68505', 'request_id': 'req-8a3ef867-38a6-4504-a4c4-34a5280bc342'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:05.278474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8a3ef867-38a6-4504-a4c4-34a5280bc342 admin admin] The token cache is being invalidate because project ea145e4b563e4092824de0f316f68505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:05.278824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8a3ef867-38a6-4504-a4c4-34a5280bc342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:05.285888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8a3ef867-38a6-4504-a4c4-34a5280bc342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:05.285888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 984/1975] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:04 2022] DELETE /identity/v3/projects/ea145e4b563e4092824de0f316f68505 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:05.294082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d02754-a7ca-47e3-8b6c-ff441e5c7596 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:05.294082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d02754-a7ca-47e3-8b6c-ff441e5c7596 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:05.294082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d02754-a7ca-47e3-8b6c-ff441e5c7596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:05.316178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-89d02754-a7ca-47e3-8b6c-ff441e5c7596 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:05.317423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-89d02754-a7ca-47e3-8b6c-ff441e5c7596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:05.386263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 992/1976] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:05.396344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7713585-f798-4505-ba9c-204c57592b93 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:05.396646 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7713585-f798-4505-ba9c-204c57592b93 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:05.396921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7713585-f798-4505-ba9c-204c57592b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:05.427079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c7713585-f798-4505-ba9c-204c57592b93 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:05.428257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d02754-a7ca-47e3-8b6c-ff441e5c7596 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:05.429003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c7713585-f798-4505-ba9c-204c57592b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:05.429883 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-89d02754-a7ca-47e3-8b6c-ff441e5c7596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:05.442917 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d02754-a7ca-47e3-8b6c-ff441e5c7596 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:05.444761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 985/1977] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:05.449285 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134cb63-132a-4621-97f2-6095562f1016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:05.449943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134cb63-132a-4621-97f2-6095562f1016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:05.450222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7713585-f798-4505-ba9c-204c57592b93 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:05.450390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134cb63-132a-4621-97f2-6095562f1016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:05.450549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134cb63-132a-4621-97f2-6095562f1016 admin admin] PATH_INFO: `/v3/users/b7bbf2a792da49bcab61f673ced3c779` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:05.451849 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134cb63-132a-4621-97f2-6095562f1016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7bbf2a792da49bcab61f673ced3c779)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:05.453901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134cb63-132a-4621-97f2-6095562f1016 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:05.462835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7713585-f798-4505-ba9c-204c57592b93 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:05.470023 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba3b81-0efc-47e5-8d66-0006f89ef3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:05.470786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba3b81-0efc-47e5-8d66-0006f89ef3cf admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:05.471155 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba3b81-0efc-47e5-8d66-0006f89ef3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:05.471641 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba3b81-0efc-47e5-8d66-0006f89ef3cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:05.472306 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba3b81-0efc-47e5-8d66-0006f89ef3cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:48:05.472704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba3b81-0efc-47e5-8d66-0006f89ef3cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:05.474497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba3b81-0efc-47e5-8d66-0006f89ef3cf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:05.489117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 986/1978] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:48:05.494641 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba3b81-0efc-47e5-8d66-0006f89ef3cf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:05.508209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba3b81-0efc-47e5-8d66-0006f89ef3cf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:05.513789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcb2e8-93d7-4874-9818-69a79d8ab597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:05.514507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcb2e8-93d7-4874-9818-69a79d8ab597 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:05.520505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcb2e8-93d7-4874-9818-69a79d8ab597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:05.520505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcb2e8-93d7-4874-9818-69a79d8ab597 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:05.520505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcb2e8-93d7-4874-9818-69a79d8ab597 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:05.520505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcb2e8-93d7-4874-9818-69a79d8ab597 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:05.683058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 987/1979] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:05 2022] POST /identity/v3/projects => generated 383 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:05.687666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcb2e8-93d7-4874-9818-69a79d8ab597 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:05.740988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3134cb63-132a-4621-97f2-6095562f1016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7bbf2a792da49bcab61f673ced3c779'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:05.743814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3134cb63-132a-4621-97f2-6095562f1016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7bbf2a792da49bcab61f673ced3c779'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:05.810962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcb2e8-93d7-4874-9818-69a79d8ab597 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:05.816050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468d1c6-2556-4103-aa20-14e72a290ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:05.816685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468d1c6-2556-4103-aa20-14e72a290ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:05.816964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468d1c6-2556-4103-aa20-14e72a290ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:05.817238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468d1c6-2556-4103-aa20-14e72a290ee1 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:05.817899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468d1c6-2556-4103-aa20-14e72a290ee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:05.819963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468d1c6-2556-4103-aa20-14e72a290ee1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:05.846439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3134cb63-132a-4621-97f2-6095562f1016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7bbf2a792da49bcab61f673ced3c779'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:05.868050 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3134cb63-132a-4621-97f2-6095562f1016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7bbf2a792da49bcab61f673ced3c779'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:05.892648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3134cb63-132a-4621-97f2-6095562f1016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7bbf2a792da49bcab61f673ced3c779'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:05.901787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 993/1980] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:05 2022] DELETE /identity/v3/users/b7bbf2a792da49bcab61f673ced3c779 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:05.906412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6b93d-2f15-4cbb-af98-ac7924839afb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:05.906700 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6b93d-2f15-4cbb-af98-ac7924839afb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:05.906983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6b93d-2f15-4cbb-af98-ac7924839afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:05.920168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 988/1981] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:05 2022] POST /identity/v3/users => generated 292 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:48:05.927607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468d1c6-2556-4103-aa20-14e72a290ee1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:05.929575 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8468d1c6-2556-4103-aa20-14e72a290ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:05.935616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-50c6b93d-2f15-4cbb-af98-ac7924839afb None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:05.937622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-50c6b93d-2f15-4cbb-af98-ac7924839afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:05.942898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468d1c6-2556-4103-aa20-14e72a290ee1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:05.994021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:06.011939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dac7c-2484-4e38-a624-140a2455e1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:06.012940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dac7c-2484-4e38-a624-140a2455e1ef admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:06.013460 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dac7c-2484-4e38-a624-140a2455e1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:06.021730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dac7c-2484-4e38-a624-140a2455e1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:06.021730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dac7c-2484-4e38-a624-140a2455e1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:06.021730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dac7c-2484-4e38-a624-140a2455e1ef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:06.038816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 989/1982] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:48:06.039151 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 994/1983] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:06.040602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dac7c-2484-4e38-a624-140a2455e1ef admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:06.049629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6b93d-2f15-4cbb-af98-ac7924839afb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:06.051291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-50c6b93d-2f15-4cbb-af98-ac7924839afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:06.067896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dac7c-2484-4e38-a624-140a2455e1ef admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:06.067896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b781a6-3370-4536-b324-27d55e161736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:06.068479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b781a6-3370-4536-b324-27d55e161736 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:06.068896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b781a6-3370-4536-b324-27d55e161736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:06.069171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b781a6-3370-4536-b324-27d55e161736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:06.070884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b781a6-3370-4536-b324-27d55e161736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:06.073958 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b781a6-3370-4536-b324-27d55e161736 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:06.094776 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 990/1984] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:06.095947 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b781a6-3370-4536-b324-27d55e161736 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:06.107915 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b781a6-3370-4536-b324-27d55e161736 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:06.117168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3017715-27f0-44af-8db1-2e6798ca0891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:06.119306 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3017715-27f0-44af-8db1-2e6798ca0891 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:06.119306 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3017715-27f0-44af-8db1-2e6798ca0891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:06.119306 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3017715-27f0-44af-8db1-2e6798ca0891 admin admin] PATH_INFO: `/v3/projects/ae7b0cf60e8a4ccab7ee630825285315/users/f2968130a5264d30a5c97ce7137f6ee8/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:06.131811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:06.132040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6b93d-2f15-4cbb-af98-ac7924839afb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:06.135768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11672c0b-94a6-45f1-8d02-fceb77fd7889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:06.136497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11672c0b-94a6-45f1-8d02-fceb77fd7889 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:06.136632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3017715-27f0-44af-8db1-2e6798ca0891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7b0cf60e8a4ccab7ee630825285315, user_id=f2968130a5264d30a5c97ce7137f6ee8, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:06.136890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11672c0b-94a6-45f1-8d02-fceb77fd7889 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:06.137294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11672c0b-94a6-45f1-8d02-fceb77fd7889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:06.139291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3017715-27f0-44af-8db1-2e6798ca0891 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:06.147138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11672c0b-94a6-45f1-8d02-fceb77fd7889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:06.153487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11672c0b-94a6-45f1-8d02-fceb77fd7889 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:06.159642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 995/1985] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:06.305872 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 991/1986] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:06 2022] PUT /identity/v3/projects/ae7b0cf60e8a4ccab7ee630825285315/users/f2968130a5264d30a5c97ce7137f6ee8/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:06.327729 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a5291-a37f-4467-a401-4c8950b53b63 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:06.328257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a5291-a37f-4467-a401-4c8950b53b63 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:06.328598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a5291-a37f-4467-a401-4c8950b53b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:06.380002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-815a5291-a37f-4467-a401-4c8950b53b63 None None] MFA Rules not processed for user `f2968130a5264d30a5c97ce7137f6ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:06.381481 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-815a5291-a37f-4467-a401-4c8950b53b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:06.479123 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 996/1987] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:06.495721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3017715-27f0-44af-8db1-2e6798ca0891 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:06.498881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c3017715-27f0-44af-8db1-2e6798ca0891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:06.583452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3017715-27f0-44af-8db1-2e6798ca0891 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:06.587818 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86910d-fe27-4824-81b5-49cb27221573 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:06.588597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86910d-fe27-4824-81b5-49cb27221573 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:06.588883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86910d-fe27-4824-81b5-49cb27221573 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:06.589156 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86910d-fe27-4824-81b5-49cb27221573 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:06.599847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86910d-fe27-4824-81b5-49cb27221573 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:06.601336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86910d-fe27-4824-81b5-49cb27221573 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:06.612689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 992/1988] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:06.634752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a5291-a37f-4467-a401-4c8950b53b63 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:06.677372 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86910d-fe27-4824-81b5-49cb27221573 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:06.680157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3f86910d-fe27-4824-81b5-49cb27221573 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:06.784406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a5291-a37f-4467-a401-4c8950b53b63 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:06.798816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc464a1-4f41-4a79-a45b-eb1bddaaea3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:06.801817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc464a1-4f41-4a79-a45b-eb1bddaaea3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:06.802452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc464a1-4f41-4a79-a45b-eb1bddaaea3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:06.803407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc464a1-4f41-4a79-a45b-eb1bddaaea3c admin admin] PATH_INFO: `/v3/users/f2968130a5264d30a5c97ce7137f6ee8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:06.806411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc464a1-4f41-4a79-a45b-eb1bddaaea3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2968130a5264d30a5c97ce7137f6ee8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:06.812024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc464a1-4f41-4a79-a45b-eb1bddaaea3c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:06.826339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86910d-fe27-4824-81b5-49cb27221573 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:06.839086 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:06.840293 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:06.841438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:06.841933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] PATH_INFO: `/v3/projects/d22f00fe9e08438a81b9a3fa2bd03a93` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:06.844735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d22f00fe9e08438a81b9a3fa2bd03a93)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:06.853732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:07.111315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:07.125103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd22f00fe9e08438a81b9a3fa2bd03a93', 'request_id': 'req-0365e99f-28cd-4493-9004-5b96d9f1742d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.127504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fc464a1-4f41-4a79-a45b-eb1bddaaea3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2968130a5264d30a5c97ce7137f6ee8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.128449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fc464a1-4f41-4a79-a45b-eb1bddaaea3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2968130a5264d30a5c97ce7137f6ee8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.274779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd22f00fe9e08438a81b9a3fa2bd03a93', 'request_id': 'req-0365e99f-28cd-4493-9004-5b96d9f1742d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.282506 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd22f00fe9e08438a81b9a3fa2bd03a93', 'request_id': 'req-0365e99f-28cd-4493-9004-5b96d9f1742d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.295539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] The token cache is being invalidate because project d22f00fe9e08438a81b9a3fa2bd03a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:07.295824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.296494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.298020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 993/1989] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:06 2022] DELETE /identity/v3/projects/d22f00fe9e08438a81b9a3fa2bd03a93 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:07.299547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:07.301345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:07.334674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:07.362738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fc464a1-4f41-4a79-a45b-eb1bddaaea3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2968130a5264d30a5c97ce7137f6ee8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.375445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fc464a1-4f41-4a79-a45b-eb1bddaaea3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2968130a5264d30a5c97ce7137f6ee8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.381694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365e99f-28cd-4493-9004-5b96d9f1742d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:07.390318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b3a06-c480-4bbc-9290-d662ddb2e1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:07.391282 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b3a06-c480-4bbc-9290-d662ddb2e1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:07.391762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b3a06-c480-4bbc-9290-d662ddb2e1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:07.392234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b3a06-c480-4bbc-9290-d662ddb2e1e5 admin admin] PATH_INFO: `/v3/users/a8d4ad5ae33a42b2bad82844cee4d02e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:07.394327 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b3a06-c480-4bbc-9290-d662ddb2e1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8d4ad5ae33a42b2bad82844cee4d02e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:07.397033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fc464a1-4f41-4a79-a45b-eb1bddaaea3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2968130a5264d30a5c97ce7137f6ee8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.398127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b3a06-c480-4bbc-9290-d662ddb2e1e5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:07.411284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 997/1990] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:06 2022] DELETE /identity/v3/users/f2968130a5264d30a5c97ce7137f6ee8 => generated 0 bytes in 776 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:07.411701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae3b4e-938f-4f6d-aa48-795f9c4794e2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:07.412204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae3b4e-938f-4f6d-aa48-795f9c4794e2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:07.412753 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae3b4e-938f-4f6d-aa48-795f9c4794e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:07.442677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b6ae3b4e-938f-4f6d-aa48-795f9c4794e2 None None] MFA Rules not processed for user `0b198ff6ed104068b38b993054617f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:07.444024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b6ae3b4e-938f-4f6d-aa48-795f9c4794e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:07.526231 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 998/1991] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:07.527259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae3b4e-938f-4f6d-aa48-795f9c4794e2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:07.528870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b6ae3b4e-938f-4f6d-aa48-795f9c4794e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:07.643930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae3b4e-938f-4f6d-aa48-795f9c4794e2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:07.651081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:07.651951 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:07.652419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:07.652788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] PATH_INFO: `/v3/users/d57a79d411e047c18d2ae40a69acb33c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:07.654411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d57a79d411e047c18d2ae40a69acb33c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:07.656878 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:07.691801 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d94b3a06-c480-4bbc-9290-d662ddb2e1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8d4ad5ae33a42b2bad82844cee4d02e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.721346 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d94b3a06-c480-4bbc-9290-d662ddb2e1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8d4ad5ae33a42b2bad82844cee4d02e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.837927 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d94b3a06-c480-4bbc-9290-d662ddb2e1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8d4ad5ae33a42b2bad82844cee4d02e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.839272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d94b3a06-c480-4bbc-9290-d662ddb2e1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8d4ad5ae33a42b2bad82844cee4d02e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.862739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d94b3a06-c480-4bbc-9290-d662ddb2e1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8d4ad5ae33a42b2bad82844cee4d02e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:07.876583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 994/1992] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:07 2022] DELETE /identity/v3/users/a8d4ad5ae33a42b2bad82844cee4d02e => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:07.876583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97882879-6460-4c97-b48e-af55e86c74d5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:07.876583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97882879-6460-4c97-b48e-af55e86c74d5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:07.876583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97882879-6460-4c97-b48e-af55e86c74d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:07.901291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-97882879-6460-4c97-b48e-af55e86c74d5 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:07.902755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-97882879-6460-4c97-b48e-af55e86c74d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:07.969636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:08.011201 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd57a79d411e047c18d2ae40a69acb33c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:08.012315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd57a79d411e047c18d2ae40a69acb33c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:08.043828 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 995/1993] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:08.046327 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034cf41-85bc-4ad0-8c58-8405977916da None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:08.046819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034cf41-85bc-4ad0-8c58-8405977916da None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:08.047290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034cf41-85bc-4ad0-8c58-8405977916da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:08.084192 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8034cf41-85bc-4ad0-8c58-8405977916da None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:08.087163 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8034cf41-85bc-4ad0-8c58-8405977916da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:08.101215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 996/1994] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:08.103897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034cf41-85bc-4ad0-8c58-8405977916da None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:08.106209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8034cf41-85bc-4ad0-8c58-8405977916da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:08.111799 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd57a79d411e047c18d2ae40a69acb33c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:08.132655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd57a79d411e047c18d2ae40a69acb33c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:08.163027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd57a79d411e047c18d2ae40a69acb33c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:08.176862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 999/1995] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:07 2022] DELETE /identity/v3/users/d57a79d411e047c18d2ae40a69acb33c => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:08.178964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:08.182402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:08.197534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:08.252308 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034cf41-85bc-4ad0-8c58-8405977916da None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:08.326118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1b7c5-2649-4f93-ab4b-5ed6e8cd5ce8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:08.332644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cc2af-ec33-4f13-8c3e-100cf8604b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:08.333332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cc2af-ec33-4f13-8c3e-100cf8604b36 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:08.333610 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cc2af-ec33-4f13-8c3e-100cf8604b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:08.333885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cc2af-ec33-4f13-8c3e-100cf8604b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:08.379250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf60878-ff49-4188-bf3b-2f51b1476a90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:08.379607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf60878-ff49-4188-bf3b-2f51b1476a90 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:08.380612 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf60878-ff49-4188-bf3b-2f51b1476a90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:08.380612 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf60878-ff49-4188-bf3b-2f51b1476a90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:08.468275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cc2af-ec33-4f13-8c3e-100cf8604b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:08.471286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cc2af-ec33-4f13-8c3e-100cf8604b36 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:08.497244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1000/1996] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:08.513742 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf60878-ff49-4188-bf3b-2f51b1476a90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:08.515717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf60878-ff49-4188-bf3b-2f51b1476a90 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:08.530413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 997/1997] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:08.985329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cc2af-ec33-4f13-8c3e-100cf8604b36 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:08.988390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-728cc2af-ec33-4f13-8c3e-100cf8604b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:09.020772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cc2af-ec33-4f13-8c3e-100cf8604b36 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:09.046454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:09.047571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:09.048063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:09.048701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] PATH_INFO: `/v3/projects/6517311247dd4c8784c5ac19b6c5c072` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:09.051058 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] RBAC: Authorizing `identity:delete_project(project_id=6517311247dd4c8784c5ac19b6c5c072)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:09.056238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:09.243972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6517311247dd4c8784c5ac19b6c5c072', 'request_id': 'req-a26885d8-310c-474e-bd75-de683a8136da'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:09.342592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6517311247dd4c8784c5ac19b6c5c072', 'request_id': 'req-a26885d8-310c-474e-bd75-de683a8136da'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:09.349507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:09.353188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6517311247dd4c8784c5ac19b6c5c072', 'request_id': 'req-a26885d8-310c-474e-bd75-de683a8136da'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:09.369864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] The token cache is being invalidate because project 6517311247dd4c8784c5ac19b6c5c072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:09.370183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:09.370941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:09.377403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1001/1998] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:08 2022] DELETE /identity/v3/projects/6517311247dd4c8784c5ac19b6c5c072 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:09.378971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02847c0-271d-4569-8280-4bb40a3abd59 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:09.379475 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02847c0-271d-4569-8280-4bb40a3abd59 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:09.379576 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02847c0-271d-4569-8280-4bb40a3abd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:09.388876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:09.391254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:09.429175 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a02847c0-271d-4569-8280-4bb40a3abd59 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:09.429175 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a02847c0-271d-4569-8280-4bb40a3abd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:09.545278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26885d8-310c-474e-bd75-de683a8136da admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:09.570729 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919de06-659b-4f4e-bc58-fa158570346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:09.574321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919de06-659b-4f4e-bc58-fa158570346c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:09.574820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919de06-659b-4f4e-bc58-fa158570346c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:09.575279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919de06-659b-4f4e-bc58-fa158570346c admin admin] PATH_INFO: `/v3/projects/fff5782a0d7e43c3a2d610850528fd51` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:09.578113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919de06-659b-4f4e-bc58-fa158570346c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff5782a0d7e43c3a2d610850528fd51)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:09.581272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919de06-659b-4f4e-bc58-fa158570346c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:09.611260 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 998/1999] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:09.611260 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02847c0-271d-4569-8280-4bb40a3abd59 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:09.611260 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a02847c0-271d-4569-8280-4bb40a3abd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:09.629631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02847c0-271d-4569-8280-4bb40a3abd59 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:09.638555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:09.639627 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:09.640089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:09.640567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] PATH_INFO: `/v3/projects/ae7b0cf60e8a4ccab7ee630825285315` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:09.642610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae7b0cf60e8a4ccab7ee630825285315)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:09.646424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:09.790437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9919de06-659b-4f4e-bc58-fa158570346c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff5782a0d7e43c3a2d610850528fd51', 'request_id': 'req-9919de06-659b-4f4e-bc58-fa158570346c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:09.818934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:09.829865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae7b0cf60e8a4ccab7ee630825285315', 'request_id': 'req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:09.912162 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9919de06-659b-4f4e-bc58-fa158570346c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff5782a0d7e43c3a2d610850528fd51', 'request_id': 'req-9919de06-659b-4f4e-bc58-fa158570346c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:09.922750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9919de06-659b-4f4e-bc58-fa158570346c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff5782a0d7e43c3a2d610850528fd51', 'request_id': 'req-9919de06-659b-4f4e-bc58-fa158570346c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:09.943259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9919de06-659b-4f4e-bc58-fa158570346c admin admin] The token cache is being invalidate because project fff5782a0d7e43c3a2d610850528fd51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:09.943548 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9919de06-659b-4f4e-bc58-fa158570346c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:09.944259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9919de06-659b-4f4e-bc58-fa158570346c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:09.947286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1002/2000] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:09 2022] DELETE /identity/v3/projects/fff5782a0d7e43c3a2d610850528fd51 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:09.952159 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f97678-4e26-48be-b1d6-5001ec5324d0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:09.952159 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f97678-4e26-48be-b1d6-5001ec5324d0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:09.952159 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f97678-4e26-48be-b1d6-5001ec5324d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:09.972674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-85f97678-4e26-48be-b1d6-5001ec5324d0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:09.974059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-85f97678-4e26-48be-b1d6-5001ec5324d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:10.019161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae7b0cf60e8a4ccab7ee630825285315', 'request_id': 'req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:10.033737 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae7b0cf60e8a4ccab7ee630825285315', 'request_id': 'req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:10.059622 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] The token cache is being invalidate because project ae7b0cf60e8a4ccab7ee630825285315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:10.059914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:10.060653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:10.062286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 999/2001] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:09 2022] DELETE /identity/v3/projects/ae7b0cf60e8a4ccab7ee630825285315 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:10.066091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1003/2002] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:10.066617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:10.066617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:10.067333 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db5324-637d-498d-9417-db919a12b09c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:10.067618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db5324-637d-498d-9417-db919a12b09c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:10.067890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db5324-637d-498d-9417-db919a12b09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:10.083562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd823442-8c2f-4f35-9b6f-7b7ed6dfa030 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:10.088969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:10.089162 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d2db5324-637d-498d-9417-db919a12b09c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:10.089600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:10.089877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:10.090161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] PATH_INFO: `/v3/users/bf19911159be4e46aa3b2a290f2cb638` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:10.090257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d2db5324-637d-498d-9417-db919a12b09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:10.091309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf19911159be4e46aa3b2a290f2cb638)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:10.093294 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:10.098600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1004/2003] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:10.099504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db5324-637d-498d-9417-db919a12b09c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:10.100801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d2db5324-637d-498d-9417-db919a12b09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:10.113170 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db5324-637d-498d-9417-db919a12b09c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:10.119083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ca1cd-1942-43fc-8936-adee76f0fec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:10.119502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ca1cd-1942-43fc-8936-adee76f0fec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:10.119790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ca1cd-1942-43fc-8936-adee76f0fec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:10.120074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ca1cd-1942-43fc-8936-adee76f0fec0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:10.120726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ca1cd-1942-43fc-8936-adee76f0fec0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:48:10.121022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ca1cd-1942-43fc-8936-adee76f0fec0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:10.122687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ca1cd-1942-43fc-8936-adee76f0fec0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:10.136322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1005/2004] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:48:10.137755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed527c-8405-4187-a4cc-58d1b6f81f65 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:10.138041 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed527c-8405-4187-a4cc-58d1b6f81f65 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:10.138363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed527c-8405-4187-a4cc-58d1b6f81f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:10.160352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-33ed527c-8405-4187-a4cc-58d1b6f81f65 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:10.161654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-33ed527c-8405-4187-a4cc-58d1b6f81f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:10.170444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1006/2005] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:10.172492 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed527c-8405-4187-a4cc-58d1b6f81f65 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:10.181444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed527c-8405-4187-a4cc-58d1b6f81f65 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:10.186716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a9438-c56a-4197-aec4-2a498e58fb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:10.187341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a9438-c56a-4197-aec4-2a498e58fb85 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:10.187620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a9438-c56a-4197-aec4-2a498e58fb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:10.187892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a9438-c56a-4197-aec4-2a498e58fb85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:10.188535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a9438-c56a-4197-aec4-2a498e58fb85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:10.190547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a9438-c56a-4197-aec4-2a498e58fb85 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:10.295784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf19911159be4e46aa3b2a290f2cb638'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:10.315300 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf19911159be4e46aa3b2a290f2cb638'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:10.320967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1007/2006] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:10 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:10.321926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a9438-c56a-4197-aec4-2a498e58fb85 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:10.393594 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a9438-c56a-4197-aec4-2a498e58fb85 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:10.399170 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866980a0-7f21-40fc-ac85-6eba54107ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:10.399806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866980a0-7f21-40fc-ac85-6eba54107ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:10.400103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866980a0-7f21-40fc-ac85-6eba54107ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:10.400380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866980a0-7f21-40fc-ac85-6eba54107ce7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:10.401070 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866980a0-7f21-40fc-ac85-6eba54107ce7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:48:10.401372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866980a0-7f21-40fc-ac85-6eba54107ce7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:10.403065 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866980a0-7f21-40fc-ac85-6eba54107ce7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:10.418196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1008/2007] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:48:10.419682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866980a0-7f21-40fc-ac85-6eba54107ce7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:10.430640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf19911159be4e46aa3b2a290f2cb638'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:10.442074 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf19911159be4e46aa3b2a290f2cb638'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:10.442486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866980a0-7f21-40fc-ac85-6eba54107ce7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:10.442651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf19911159be4e46aa3b2a290f2cb638'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:10.442809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12ed9-9819-4ea7-9916-d68dfedde9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:10.442809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12ed9-9819-4ea7-9916-d68dfedde9e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:10.442809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12ed9-9819-4ea7-9916-d68dfedde9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:10.443249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12ed9-9819-4ea7-9916-d68dfedde9e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:10.443910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba12ed9-9819-4ea7-9916-d68dfedde9e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:10.446960 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba12ed9-9819-4ea7-9916-d68dfedde9e8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:10.449356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1000/2008] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:10 2022] DELETE /identity/v3/users/bf19911159be4e46aa3b2a290f2cb638 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:10.455296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:10.457414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:10.568034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892ca9a-7cc5-4b1d-8386-288150f4e1a0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:10.574233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aca57b-b841-4f6d-a483-bb42768c5aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:10.574928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aca57b-b841-4f6d-a483-bb42768c5aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:10.575113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aca57b-b841-4f6d-a483-bb42768c5aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:10.577416 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aca57b-b841-4f6d-a483-bb42768c5aa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:10.577416 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aca57b-b841-4f6d-a483-bb42768c5aa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:10.578321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aca57b-b841-4f6d-a483-bb42768c5aa4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:10.585408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1009/2009] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:10 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:10.590125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12ed9-9819-4ea7-9916-d68dfedde9e8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:10.593134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cba12ed9-9819-4ea7-9916-d68dfedde9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:10.607316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12ed9-9819-4ea7-9916-d68dfedde9e8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:10.614357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130dba9-2461-439c-805c-12a94aa5a68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:10.615035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130dba9-2461-439c-805c-12a94aa5a68d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:10.615370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130dba9-2461-439c-805c-12a94aa5a68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:10.615582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130dba9-2461-439c-805c-12a94aa5a68d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:10.616922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130dba9-2461-439c-805c-12a94aa5a68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:10.618870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130dba9-2461-439c-805c-12a94aa5a68d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:10.632273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1010/2010] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:48:10.638455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130dba9-2461-439c-805c-12a94aa5a68d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:10.658071 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130dba9-2461-439c-805c-12a94aa5a68d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:10.664471 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1793fb0-fad2-45dc-8054-6ae6e80edab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:10.665139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1793fb0-fad2-45dc-8054-6ae6e80edab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:10.665383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1793fb0-fad2-45dc-8054-6ae6e80edab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:10.665653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1793fb0-fad2-45dc-8054-6ae6e80edab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:10.666442 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1793fb0-fad2-45dc-8054-6ae6e80edab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:10.668067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1793fb0-fad2-45dc-8054-6ae6e80edab7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:10.682206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1011/2011] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:10.686008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1793fb0-fad2-45dc-8054-6ae6e80edab7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:10.698639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1793fb0-fad2-45dc-8054-6ae6e80edab7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:10.706729 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bba21-4751-4ee0-9d44-517db0c9acad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:10.707480 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bba21-4751-4ee0-9d44-517db0c9acad admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:10.707829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bba21-4751-4ee0-9d44-517db0c9acad admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:10.708176 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bba21-4751-4ee0-9d44-517db0c9acad admin admin] PATH_INFO: `/v3/projects/4a20f4de05b84a1e84cfeaff7057be37/users/863637528a124de4b98c6b1b1b78c3be/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:10.725485 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bba21-4751-4ee0-9d44-517db0c9acad admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a20f4de05b84a1e84cfeaff7057be37, user_id=863637528a124de4b98c6b1b1b78c3be, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:10.729009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bba21-4751-4ee0-9d44-517db0c9acad admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:10.741784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1001/2012] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:10 2022] POST /identity/v3/projects => generated 373 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:10.746180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aca57b-b841-4f6d-a483-bb42768c5aa4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:10.846207 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aca57b-b841-4f6d-a483-bb42768c5aa4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:10.867366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1012/2013] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:10 2022] PUT /identity/v3/projects/4a20f4de05b84a1e84cfeaff7057be37/users/863637528a124de4b98c6b1b1b78c3be/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:10.871202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fec190-1431-4ff0-81e1-5789353e80a1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:10.871672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fec190-1431-4ff0-81e1-5789353e80a1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:10.871956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fec190-1431-4ff0-81e1-5789353e80a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:10.922629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a6fec190-1431-4ff0-81e1-5789353e80a1 None None] MFA Rules not processed for user `863637528a124de4b98c6b1b1b78c3be`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:10.924009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a6fec190-1431-4ff0-81e1-5789353e80a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:11.000345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6bee8-b7bc-48e8-acba-d20d9c22c285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:11.001383 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6bee8-b7bc-48e8-acba-d20d9c22c285 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:11.004039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6bee8-b7bc-48e8-acba-d20d9c22c285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:11.004039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6bee8-b7bc-48e8-acba-d20d9c22c285 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:11.004039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6bee8-b7bc-48e8-acba-d20d9c22c285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:11.007977 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6bee8-b7bc-48e8-acba-d20d9c22c285 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:11.047720 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1013/2014] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:11.060321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fec190-1431-4ff0-81e1-5789353e80a1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:11.061896 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a6fec190-1431-4ff0-81e1-5789353e80a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:11.134043 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1002/2015] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:10 2022] POST /identity/v3/users => generated 287 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:48:11.134043 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6bee8-b7bc-48e8-acba-d20d9c22c285 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:11.137952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c3d6bee8-b7bc-48e8-acba-d20d9c22c285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:11.161827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6bee8-b7bc-48e8-acba-d20d9c22c285 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:11.168542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:11.169651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:11.170114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:11.170598 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1 admin admin] PATH_INFO: `/v3/projects/6a139dc8eff74531863f0b9cd1248915` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:11.172238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a139dc8eff74531863f0b9cd1248915)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:11.174648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:11.184520 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fec190-1431-4ff0-81e1-5789353e80a1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:11.192476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5143e0-136a-4171-9326-b2081ba04d30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:11.194045 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5143e0-136a-4171-9326-b2081ba04d30 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:11.194676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5143e0-136a-4171-9326-b2081ba04d30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:11.195274 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5143e0-136a-4171-9326-b2081ba04d30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:11.215750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5143e0-136a-4171-9326-b2081ba04d30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:11.219124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5143e0-136a-4171-9326-b2081ba04d30 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:11.236800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1014/2016] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:48:11.238326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5143e0-136a-4171-9326-b2081ba04d30 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:11.254503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:11.257525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5143e0-136a-4171-9326-b2081ba04d30 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:11.267218 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97dc11-28c9-4943-9265-b53312f40497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:11.268243 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97dc11-28c9-4943-9265-b53312f40497 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:11.268814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97dc11-28c9-4943-9265-b53312f40497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:11.269158 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97dc11-28c9-4943-9265-b53312f40497 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:11.270318 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97dc11-28c9-4943-9265-b53312f40497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:11.274128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97dc11-28c9-4943-9265-b53312f40497 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:11.294282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1015/2017] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:11.298965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97dc11-28c9-4943-9265-b53312f40497 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:11.317124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97dc11-28c9-4943-9265-b53312f40497 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:11.339814 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a139dc8eff74531863f0b9cd1248915', 'request_id': 'req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:11.387993 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:11.429574 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516e595-4cee-412a-b0f6-98d98c47df30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:11.436262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516e595-4cee-412a-b0f6-98d98c47df30 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:11.436262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516e595-4cee-412a-b0f6-98d98c47df30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:11.436262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516e595-4cee-412a-b0f6-98d98c47df30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:11.436262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516e595-4cee-412a-b0f6-98d98c47df30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:11.436262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516e595-4cee-412a-b0f6-98d98c47df30 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:11.440284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a139dc8eff74531863f0b9cd1248915', 'request_id': 'req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:11.450829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1016/2018] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:48:11.452502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516e595-4cee-412a-b0f6-98d98c47df30 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:11.453872 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a139dc8eff74531863f0b9cd1248915', 'request_id': 'req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:11.472907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516e595-4cee-412a-b0f6-98d98c47df30 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:11.481406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1 admin admin] The token cache is being invalidate because project 6a139dc8eff74531863f0b9cd1248915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:11.481896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:11.485802 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-49fc5044-4e01-48bf-9c68-14ac24bd4fd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:11.486017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec6368-27aa-4e44-aba5-6e79cd3ec0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:11.486408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec6368-27aa-4e44-aba5-6e79cd3ec0c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:11.486566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1003/2019] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:11 2022] DELETE /identity/v3/projects/6a139dc8eff74531863f0b9cd1248915 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:11.486876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec6368-27aa-4e44-aba5-6e79cd3ec0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:11.487340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec6368-27aa-4e44-aba5-6e79cd3ec0c8 admin admin] PATH_INFO: `/v3/projects/258285eb5a2c4d9eb8187567a5b3515d/users/52063cee29f142d780f5a177904ba056/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:11.492233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5b7a7-39d1-4b9b-bd3f-5bde0208b0af None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:11.493796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5b7a7-39d1-4b9b-bd3f-5bde0208b0af None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:11.494459 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5b7a7-39d1-4b9b-bd3f-5bde0208b0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:11.508171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec6368-27aa-4e44-aba5-6e79cd3ec0c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=258285eb5a2c4d9eb8187567a5b3515d, user_id=52063cee29f142d780f5a177904ba056, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:11.511332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec6368-27aa-4e44-aba5-6e79cd3ec0c8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:11.528661 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6ea5b7a7-39d1-4b9b-bd3f-5bde0208b0af None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:11.530635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ea5b7a7-39d1-4b9b-bd3f-5bde0208b0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:11.542818 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1004/2020] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:11.551287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cbce3-73d5-4d6d-9ebe-44438effebf8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:11.551561 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cbce3-73d5-4d6d-9ebe-44438effebf8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:11.551838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cbce3-73d5-4d6d-9ebe-44438effebf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:11.575802 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4c2cbce3-73d5-4d6d-9ebe-44438effebf8 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:11.578190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4c2cbce3-73d5-4d6d-9ebe-44438effebf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:11.594469 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1005/2021] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:11.599153 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cbce3-73d5-4d6d-9ebe-44438effebf8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:11.610023 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cbce3-73d5-4d6d-9ebe-44438effebf8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:11.615904 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e92384-3213-4606-9133-866e4d6eb8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:11.616773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e92384-3213-4606-9133-866e4d6eb8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:11.617256 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e92384-3213-4606-9133-866e4d6eb8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:11.617642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e92384-3213-4606-9133-866e4d6eb8d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:11.618557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e92384-3213-4606-9133-866e4d6eb8d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:48:11.619063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e92384-3213-4606-9133-866e4d6eb8d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:11.620969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e92384-3213-4606-9133-866e4d6eb8d4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:11.639118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1006/2022] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:48:11.645491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e92384-3213-4606-9133-866e4d6eb8d4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:11.653860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1017/2023] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:11 2022] PUT /identity/v3/projects/258285eb5a2c4d9eb8187567a5b3515d/users/52063cee29f142d780f5a177904ba056/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:11.664486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b9ace-3f7c-433d-8dc5-7e0652693c7e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:11.664790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b9ace-3f7c-433d-8dc5-7e0652693c7e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:11.665038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b9ace-3f7c-433d-8dc5-7e0652693c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:11.666586 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e92384-3213-4606-9133-866e4d6eb8d4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:11.713849 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9d1b9ace-3f7c-433d-8dc5-7e0652693c7e None None] MFA Rules not processed for user `52063cee29f142d780f5a177904ba056`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:11.716188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9d1b9ace-3f7c-433d-8dc5-7e0652693c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:11.777353 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834470c-a05a-41cb-8c43-46cfa317391c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:11.778473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834470c-a05a-41cb-8c43-46cfa317391c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:11.778973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834470c-a05a-41cb-8c43-46cfa317391c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:11.779456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834470c-a05a-41cb-8c43-46cfa317391c admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:11.780665 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834470c-a05a-41cb-8c43-46cfa317391c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:11.784522 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834470c-a05a-41cb-8c43-46cfa317391c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:11.824883 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1018/2024] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:11.854135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b9ace-3f7c-433d-8dc5-7e0652693c7e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:11.862678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9d1b9ace-3f7c-433d-8dc5-7e0652693c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:11.970229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b9ace-3f7c-433d-8dc5-7e0652693c7e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:11.979972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9acbe-88b2-47e7-a139-d16798e8fb9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:11.980776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9acbe-88b2-47e7-a139-d16798e8fb9e service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:11.983779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9acbe-88b2-47e7-a139-d16798e8fb9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:11.983779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9acbe-88b2-47e7-a139-d16798e8fb9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:11.988963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1007/2025] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:11 2022] POST /identity/v3/projects => generated 381 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:48:11.994601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9acbe-88b2-47e7-a139-d16798e8fb9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:11.994850 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834470c-a05a-41cb-8c43-46cfa317391c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:11.996260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9acbe-88b2-47e7-a139-d16798e8fb9e service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:12.011593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1019/2026] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:12.069914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9acbe-88b2-47e7-a139-d16798e8fb9e service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:12.073135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-10f9acbe-88b2-47e7-a139-d16798e8fb9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:12.096442 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9acbe-88b2-47e7-a139-d16798e8fb9e service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:12.149369 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834470c-a05a-41cb-8c43-46cfa317391c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:12.156551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efbf45-f898-4075-be1b-0cb8ef372066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:12.158106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efbf45-f898-4075-be1b-0cb8ef372066 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:12.158106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efbf45-f898-4075-be1b-0cb8ef372066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:12.158106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efbf45-f898-4075-be1b-0cb8ef372066 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:12.158976 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efbf45-f898-4075-be1b-0cb8ef372066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:12.161374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efbf45-f898-4075-be1b-0cb8ef372066 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:12.223366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871a809-8877-477f-8fbe-50b711894a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:12.224428 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871a809-8877-477f-8fbe-50b711894a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:12.226786 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871a809-8877-477f-8fbe-50b711894a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:12.226786 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871a809-8877-477f-8fbe-50b711894a39 admin admin] PATH_INFO: `/v3/users/52063cee29f142d780f5a177904ba056` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:12.228114 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871a809-8877-477f-8fbe-50b711894a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52063cee29f142d780f5a177904ba056)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:12.232080 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871a809-8877-477f-8fbe-50b711894a39 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:12.274206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1008/2027] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:11 2022] POST /identity/v3/users => generated 291 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:12.279040 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efbf45-f898-4075-be1b-0cb8ef372066 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:12.291868 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efbf45-f898-4075-be1b-0cb8ef372066 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:12.297379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10ebf7-c31d-4b62-ac1e-a2120cbca84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:12.298117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10ebf7-c31d-4b62-ac1e-a2120cbca84d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:12.298410 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10ebf7-c31d-4b62-ac1e-a2120cbca84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:12.298681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10ebf7-c31d-4b62-ac1e-a2120cbca84d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:12.299350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10ebf7-c31d-4b62-ac1e-a2120cbca84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:12.301820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10ebf7-c31d-4b62-ac1e-a2120cbca84d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:12.315537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1009/2028] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:12.319403 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10ebf7-c31d-4b62-ac1e-a2120cbca84d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:12.331835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10ebf7-c31d-4b62-ac1e-a2120cbca84d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:12.338191 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d3b92-c07a-4deb-96e2-da6ded37caeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:12.339114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d3b92-c07a-4deb-96e2-da6ded37caeb admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:12.339400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d3b92-c07a-4deb-96e2-da6ded37caeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:12.339694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d3b92-c07a-4deb-96e2-da6ded37caeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:12.340351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d3b92-c07a-4deb-96e2-da6ded37caeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:12.341999 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d3b92-c07a-4deb-96e2-da6ded37caeb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:12.357132 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1010/2029] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:48:12.362216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d3b92-c07a-4deb-96e2-da6ded37caeb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:12.393315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d3b92-c07a-4deb-96e2-da6ded37caeb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:12.403748 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275379a-6285-41ff-830e-50b6cfd4fb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:12.404888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275379a-6285-41ff-830e-50b6cfd4fb60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:12.405882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275379a-6285-41ff-830e-50b6cfd4fb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:12.406071 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275379a-6285-41ff-830e-50b6cfd4fb60 admin admin] PATH_INFO: `/v3/projects/0321961b0c044d9cb899ac92d1771422/users/157cc3f324004773a9d3f5d88cc955ed/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:12.436759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275379a-6285-41ff-830e-50b6cfd4fb60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0321961b0c044d9cb899ac92d1771422, user_id=157cc3f324004773a9d3f5d88cc955ed, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:12.440115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275379a-6285-41ff-830e-50b6cfd4fb60 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:12.517120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e871a809-8877-477f-8fbe-50b711894a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52063cee29f142d780f5a177904ba056'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:12.517784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e871a809-8877-477f-8fbe-50b711894a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52063cee29f142d780f5a177904ba056'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:12.536673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:12.559188 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1011/2030] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:12 2022] PUT /identity/v3/projects/0321961b0c044d9cb899ac92d1771422/users/157cc3f324004773a9d3f5d88cc955ed/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:12.573919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be0f44-216a-40b0-9420-ded45f761d3b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:12.574447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be0f44-216a-40b0-9420-ded45f761d3b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:12.574728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be0f44-216a-40b0-9420-ded45f761d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:12.611564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e3be0f44-216a-40b0-9420-ded45f761d3b None None] MFA Rules not processed for user `157cc3f324004773a9d3f5d88cc955ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:12.613423 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e3be0f44-216a-40b0-9420-ded45f761d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:12.632328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e871a809-8877-477f-8fbe-50b711894a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52063cee29f142d780f5a177904ba056'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:12.643125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e871a809-8877-477f-8fbe-50b711894a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52063cee29f142d780f5a177904ba056'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:12.668944 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e871a809-8877-477f-8fbe-50b711894a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52063cee29f142d780f5a177904ba056'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:12.678236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1020/2031] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:12 2022] DELETE /identity/v3/users/52063cee29f142d780f5a177904ba056 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:12.685206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f4c8c-9737-4999-acb8-c41cac8ea8fd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:12.686825 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f4c8c-9737-4999-acb8-c41cac8ea8fd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:12.686825 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f4c8c-9737-4999-acb8-c41cac8ea8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:12.720932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-032f4c8c-9737-4999-acb8-c41cac8ea8fd None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:12.722965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-032f4c8c-9737-4999-acb8-c41cac8ea8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:12.776806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:12.785793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1012/2032] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:12.787448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be0f44-216a-40b0-9420-ded45f761d3b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:12.797438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e3be0f44-216a-40b0-9420-ded45f761d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:12.870113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1021/2033] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:12.882497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f4c8c-9737-4999-acb8-c41cac8ea8fd None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:12.885854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-032f4c8c-9737-4999-acb8-c41cac8ea8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:12.945303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be0f44-216a-40b0-9420-ded45f761d3b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:12.954340 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b75652-5b24-4a6e-8862-a3e48fdc368c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:12.955550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b75652-5b24-4a6e-8862-a3e48fdc368c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:12.956119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b75652-5b24-4a6e-8862-a3e48fdc368c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:12.956693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b75652-5b24-4a6e-8862-a3e48fdc368c admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:12.958200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b75652-5b24-4a6e-8862-a3e48fdc368c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:12.963244 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b75652-5b24-4a6e-8862-a3e48fdc368c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:12.996061 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:13.003941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f4c8c-9737-4999-acb8-c41cac8ea8fd None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:13.008695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae65685-5ccc-465a-80de-c50aa2304da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:13.009383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae65685-5ccc-465a-80de-c50aa2304da9 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:13.010653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae65685-5ccc-465a-80de-c50aa2304da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:13.011029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae65685-5ccc-465a-80de-c50aa2304da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:13.024507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:13.032754 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae65685-5ccc-465a-80de-c50aa2304da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:13.032754 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae65685-5ccc-465a-80de-c50aa2304da9 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:13.043358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1022/2034] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:48:13.096663 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1013/2035] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:12 2022] POST /identity/v3/projects => generated 379 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:13.098463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae65685-5ccc-465a-80de-c50aa2304da9 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:13.112419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae65685-5ccc-465a-80de-c50aa2304da9 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:13.233641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81895b8-d404-4bc5-b382-5932742723c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:13.234777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81895b8-d404-4bc5-b382-5932742723c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:13.235665 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81895b8-d404-4bc5-b382-5932742723c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:13.235665 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81895b8-d404-4bc5-b382-5932742723c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:13.236239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81895b8-d404-4bc5-b382-5932742723c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:13.239366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81895b8-d404-4bc5-b382-5932742723c1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:13.288901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:13.401704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1023/2036] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:13 2022] POST /identity/v3/users => generated 296 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:13.405544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b75652-5b24-4a6e-8862-a3e48fdc368c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:13.425646 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b75652-5b24-4a6e-8862-a3e48fdc368c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:13.449028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a426d1-aff3-45e1-b1d7-5ad3f0d6b3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:13.452867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a426d1-aff3-45e1-b1d7-5ad3f0d6b3cf admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:13.454425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a426d1-aff3-45e1-b1d7-5ad3f0d6b3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:13.454867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a426d1-aff3-45e1-b1d7-5ad3f0d6b3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:13.455674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a426d1-aff3-45e1-b1d7-5ad3f0d6b3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:13.458307 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a426d1-aff3-45e1-b1d7-5ad3f0d6b3cf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:13.483558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1014/2037] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:13.489916 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81895b8-d404-4bc5-b382-5932742723c1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:13.563095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81895b8-d404-4bc5-b382-5932742723c1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:13.603241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d10605-9ace-4609-9cb1-970d69eb3257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:13.603241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d10605-9ace-4609-9cb1-970d69eb3257 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:13.615065 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d10605-9ace-4609-9cb1-970d69eb3257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:13.616595 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d10605-9ace-4609-9cb1-970d69eb3257 admin admin] PATH_INFO: `/v3/projects/8dd0fb04a9ce4aedbf7218cc893df256/users/9135da1f04aa4000982d44c15e73e46c/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:13.641184 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d10605-9ace-4609-9cb1-970d69eb3257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd0fb04a9ce4aedbf7218cc893df256, user_id=9135da1f04aa4000982d44c15e73e46c, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:13.644218 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d10605-9ace-4609-9cb1-970d69eb3257 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:13.843049 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1024/2038] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:13 2022] PUT /identity/v3/projects/8dd0fb04a9ce4aedbf7218cc893df256/users/9135da1f04aa4000982d44c15e73e46c/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:13.848681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a426d1-aff3-45e1-b1d7-5ad3f0d6b3cf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:13.854209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d10605-9ace-4609-9cb1-970d69eb3257 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:13.858825 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d2d10605-9ace-4609-9cb1-970d69eb3257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:13.867283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a426d1-aff3-45e1-b1d7-5ad3f0d6b3cf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:14.016101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d10605-9ace-4609-9cb1-970d69eb3257 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:14.027487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f519bb6-c00b-4ba9-a905-733f0d1c040e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:14.027962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f519bb6-c00b-4ba9-a905-733f0d1c040e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:14.028431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f519bb6-c00b-4ba9-a905-733f0d1c040e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:14.028966 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f519bb6-c00b-4ba9-a905-733f0d1c040e admin admin] PATH_INFO: `/v3/projects/258285eb5a2c4d9eb8187567a5b3515d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:14.031765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f519bb6-c00b-4ba9-a905-733f0d1c040e admin admin] RBAC: Authorizing `identity:delete_project(project_id=258285eb5a2c4d9eb8187567a5b3515d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:14.043240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ac877-7636-4301-8a07-447fa08257b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:14.043772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f519bb6-c00b-4ba9-a905-733f0d1c040e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:14.045076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ac877-7636-4301-8a07-447fa08257b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:14.045773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ac877-7636-4301-8a07-447fa08257b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:14.046560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ac877-7636-4301-8a07-447fa08257b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:14.047981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ac877-7636-4301-8a07-447fa08257b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:14.052351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ac877-7636-4301-8a07-447fa08257b8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:14.078476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1015/2039] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:14.081936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ac877-7636-4301-8a07-447fa08257b8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:14.104613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ac877-7636-4301-8a07-447fa08257b8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:14.113805 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87439205-390c-4be4-8ce8-0149a6cab830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:14.114993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87439205-390c-4be4-8ce8-0149a6cab830 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:14.115320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87439205-390c-4be4-8ce8-0149a6cab830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:14.115692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87439205-390c-4be4-8ce8-0149a6cab830 admin admin] PATH_INFO: `/v3/projects/8dd0fb04a9ce4aedbf7218cc893df256/users/9135da1f04aa4000982d44c15e73e46c/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:14.118800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87439205-390c-4be4-8ce8-0149a6cab830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd0fb04a9ce4aedbf7218cc893df256, user_id=9135da1f04aa4000982d44c15e73e46c, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:14.124176 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87439205-390c-4be4-8ce8-0149a6cab830 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:14.125214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:14.142068 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1016/2040] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:14 2022] PUT /identity/v3/projects/8dd0fb04a9ce4aedbf7218cc893df256/users/9135da1f04aa4000982d44c15e73e46c/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:14.154376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2804d-7717-46e0-a2db-02527ad3871b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:14.154670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2804d-7717-46e0-a2db-02527ad3871b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:14.154918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2804d-7717-46e0-a2db-02527ad3871b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:14.189288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dbf2804d-7717-46e0-a2db-02527ad3871b None None] MFA Rules not processed for user `9135da1f04aa4000982d44c15e73e46c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:14.190756 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dbf2804d-7717-46e0-a2db-02527ad3871b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:14.228060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:14.251961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5f519bb6-c00b-4ba9-a905-733f0d1c040e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '258285eb5a2c4d9eb8187567a5b3515d', 'request_id': 'req-5f519bb6-c00b-4ba9-a905-733f0d1c040e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:14.345208 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1017/2041] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:14 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:14.345402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5f519bb6-c00b-4ba9-a905-733f0d1c040e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '258285eb5a2c4d9eb8187567a5b3515d', 'request_id': 'req-5f519bb6-c00b-4ba9-a905-733f0d1c040e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:14.358202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5f519bb6-c00b-4ba9-a905-733f0d1c040e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '258285eb5a2c4d9eb8187567a5b3515d', 'request_id': 'req-5f519bb6-c00b-4ba9-a905-733f0d1c040e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:14.361396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2804d-7717-46e0-a2db-02527ad3871b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:14.362851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dbf2804d-7717-46e0-a2db-02527ad3871b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:14.374809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5f519bb6-c00b-4ba9-a905-733f0d1c040e admin admin] The token cache is being invalidate because project 258285eb5a2c4d9eb8187567a5b3515d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:14.375324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5f519bb6-c00b-4ba9-a905-733f0d1c040e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:14.376622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5f519bb6-c00b-4ba9-a905-733f0d1c040e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:14.380200 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1025/2042] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:13 2022] DELETE /identity/v3/projects/258285eb5a2c4d9eb8187567a5b3515d => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:14.385306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c48e-27c6-4361-a0ac-b012b08bb878 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:14.385639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c48e-27c6-4361-a0ac-b012b08bb878 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:14.391166 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c48e-27c6-4361-a0ac-b012b08bb878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:14.423214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f1c4c48e-27c6-4361-a0ac-b012b08bb878 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:14.424588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f1c4c48e-27c6-4361-a0ac-b012b08bb878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:14.512560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2804d-7717-46e0-a2db-02527ad3871b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:14.556294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1026/2043] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:14.566276 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a45f8-8550-4467-ab5e-63cf649aa7b4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:14.570311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a45f8-8550-4467-ab5e-63cf649aa7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:14.570311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a45f8-8550-4467-ab5e-63cf649aa7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:14.593157 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-444a45f8-8550-4467-ab5e-63cf649aa7b4 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:14.598340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-444a45f8-8550-4467-ab5e-63cf649aa7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:14.624961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1027/2044] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:14.625161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b26b5e-f8b7-4a8a-ae43-8dce7cd604b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:14.625161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b26b5e-f8b7-4a8a-ae43-8dce7cd604b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:14.625161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b26b5e-f8b7-4a8a-ae43-8dce7cd604b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:14.625161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b26b5e-f8b7-4a8a-ae43-8dce7cd604b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:14.626109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f0b26b5e-f8b7-4a8a-ae43-8dce7cd604b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:14.631550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a45f8-8550-4467-ab5e-63cf649aa7b4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:14.654042 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a45f8-8550-4467-ab5e-63cf649aa7b4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:14.664318 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63096e-3bd3-4b8f-9db7-dee65e2f12cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:14.665629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63096e-3bd3-4b8f-9db7-dee65e2f12cd admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:14.666184 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63096e-3bd3-4b8f-9db7-dee65e2f12cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:14.677780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63096e-3bd3-4b8f-9db7-dee65e2f12cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:14.678635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63096e-3bd3-4b8f-9db7-dee65e2f12cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:48:14.679971 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63096e-3bd3-4b8f-9db7-dee65e2f12cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:14.684513 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63096e-3bd3-4b8f-9db7-dee65e2f12cd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:14.718242 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1028/2045] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:48:14.723190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63096e-3bd3-4b8f-9db7-dee65e2f12cd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:14.742685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63096e-3bd3-4b8f-9db7-dee65e2f12cd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:14.750413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05023e84-de24-469b-9169-2d45499a4414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:14.751576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023e84-de24-469b-9169-2d45499a4414 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:14.751882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023e84-de24-469b-9169-2d45499a4414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:14.752632 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05023e84-de24-469b-9169-2d45499a4414 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:14.753341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05023e84-de24-469b-9169-2d45499a4414 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:14.756273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05023e84-de24-469b-9169-2d45499a4414 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:14.785355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b26b5e-f8b7-4a8a-ae43-8dce7cd604b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:14.788739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b26b5e-f8b7-4a8a-ae43-8dce7cd604b3 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:14.810603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1018/2046] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:48:14.873887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1029/2047] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:14 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:14.878804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b26b5e-f8b7-4a8a-ae43-8dce7cd604b3 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:14.893497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b26b5e-f8b7-4a8a-ae43-8dce7cd604b3 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:14.923486 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:14.982123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f2e18-8091-4564-b531-930de0637a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:14.983121 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f2e18-8091-4564-b531-930de0637a38 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:14.983392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f2e18-8091-4564-b531-930de0637a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:14.983679 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f2e18-8091-4564-b531-930de0637a38 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:14.984400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f2e18-8091-4564-b531-930de0637a38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:14.986764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f2e18-8091-4564-b531-930de0637a38 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:15.118186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05023e84-de24-469b-9169-2d45499a4414 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:15.120063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-05023e84-de24-469b-9169-2d45499a4414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:15.135963 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:15.136232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1019/2048] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:14 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:15.138318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f2e18-8091-4564-b531-930de0637a38 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:15.152290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f2e18-8091-4564-b531-930de0637a38 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:15.159573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e03c4-2790-440d-b08b-92831cd84bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:15.160288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e03c4-2790-440d-b08b-92831cd84bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:15.160571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e03c4-2790-440d-b08b-92831cd84bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:15.166514 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e03c4-2790-440d-b08b-92831cd84bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:15.166514 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e03c4-2790-440d-b08b-92831cd84bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:15.166514 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e03c4-2790-440d-b08b-92831cd84bc8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:15.177184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1020/2049] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:15.184107 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e03c4-2790-440d-b08b-92831cd84bc8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:15.199491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e03c4-2790-440d-b08b-92831cd84bc8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:15.202590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05023e84-de24-469b-9169-2d45499a4414 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:15.206495 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad158912-b760-4c55-b47a-951f71650b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:15.207059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77ebb0-3a76-41c0-8ad7-c44551a880c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:15.207562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad158912-b760-4c55-b47a-951f71650b99 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:15.207726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77ebb0-3a76-41c0-8ad7-c44551a880c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:15.207975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77ebb0-3a76-41c0-8ad7-c44551a880c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:15.208228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77ebb0-3a76-41c0-8ad7-c44551a880c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:15.208869 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad158912-b760-4c55-b47a-951f71650b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:15.209299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad158912-b760-4c55-b47a-951f71650b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:15.210350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad158912-b760-4c55-b47a-951f71650b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:15.210609 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-da77ebb0-3a76-41c0-8ad7-c44551a880c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:15.212464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad158912-b760-4c55-b47a-951f71650b99 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:15.233392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1021/2050] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:48:15.234850 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad158912-b760-4c55-b47a-951f71650b99 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:15.248999 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad158912-b760-4c55-b47a-951f71650b99 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:15.259095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83297676-4fe9-4696-b455-ffd99b3020e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:15.260196 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83297676-4fe9-4696-b455-ffd99b3020e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:15.260704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83297676-4fe9-4696-b455-ffd99b3020e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:15.261198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83297676-4fe9-4696-b455-ffd99b3020e5 admin admin] PATH_INFO: `/v3/projects/b4f426194c8d41bcaee7862c5d04b180/users/a0dde6aa81204252b66b6e640b343ac3/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:15.289693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83297676-4fe9-4696-b455-ffd99b3020e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f426194c8d41bcaee7862c5d04b180, user_id=a0dde6aa81204252b66b6e640b343ac3, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:15.296851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77ebb0-3a76-41c0-8ad7-c44551a880c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:15.296851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77ebb0-3a76-41c0-8ad7-c44551a880c6 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:15.297475 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83297676-4fe9-4696-b455-ffd99b3020e5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:15.304998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1030/2051] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:48:15.394944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1022/2052] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:15 2022] PUT /identity/v3/projects/b4f426194c8d41bcaee7862c5d04b180/users/a0dde6aa81204252b66b6e640b343ac3/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:15.405881 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212d5ad-890d-4e6c-80d9-c14212640a81 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:15.406186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212d5ad-890d-4e6c-80d9-c14212640a81 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:15.406463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212d5ad-890d-4e6c-80d9-c14212640a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:15.427090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b212d5ad-890d-4e6c-80d9-c14212640a81 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:15.428454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b212d5ad-890d-4e6c-80d9-c14212640a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:15.505302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1031/2053] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:15.516085 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83297676-4fe9-4696-b455-ffd99b3020e5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:15.517977 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-83297676-4fe9-4696-b455-ffd99b3020e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:15.578355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83297676-4fe9-4696-b455-ffd99b3020e5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:15.582044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d78c2e-6f19-40f8-aeeb-9bbd3dc90390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:15.582677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d78c2e-6f19-40f8-aeeb-9bbd3dc90390 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:15.583037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d78c2e-6f19-40f8-aeeb-9bbd3dc90390 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:15.583312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d78c2e-6f19-40f8-aeeb-9bbd3dc90390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:15.593190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d78c2e-6f19-40f8-aeeb-9bbd3dc90390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:15.594723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d78c2e-6f19-40f8-aeeb-9bbd3dc90390 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:15.606095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1023/2054] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:23.590522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05abba5-ecd7-4524-b5bb-888d6985f4b3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:23.591718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05abba5-ecd7-4524-b5bb-888d6985f4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:23.592361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05abba5-ecd7-4524-b5bb-888d6985f4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:23.644417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a05abba5-ecd7-4524-b5bb-888d6985f4b3 None None] MFA Rules not processed for user `a0dde6aa81204252b66b6e640b343ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:23.646251 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a05abba5-ecd7-4524-b5bb-888d6985f4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:23.737677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:23.763906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1032/2055] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:23.768727 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d78c2e-6f19-40f8-aeeb-9bbd3dc90390 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:23.780298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d78c2e-6f19-40f8-aeeb-9bbd3dc90390 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:23.787591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5834095-3726-41c5-a357-f7902bde767e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:23.787591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5834095-3726-41c5-a357-f7902bde767e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:23.787591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5834095-3726-41c5-a357-f7902bde767e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:23.788042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5834095-3726-41c5-a357-f7902bde767e admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:23.788525 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5834095-3726-41c5-a357-f7902bde767e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:23.790561 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5834095-3726-41c5-a357-f7902bde767e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:24.111610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1024/2056] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:23 2022] POST /identity/v3/projects => generated 385 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:48:24.116756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05abba5-ecd7-4524-b5bb-888d6985f4b3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:24.137608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05abba5-ecd7-4524-b5bb-888d6985f4b3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:24.271115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c29a83-2bf3-495d-9a5c-851e989ea7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:24.272143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c29a83-2bf3-495d-9a5c-851e989ea7ed admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:24.273321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c29a83-2bf3-495d-9a5c-851e989ea7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:24.273750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c29a83-2bf3-495d-9a5c-851e989ea7ed admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:24.274807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c29a83-2bf3-495d-9a5c-851e989ea7ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:24.280037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c29a83-2bf3-495d-9a5c-851e989ea7ed admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:24.406467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1033/2057] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:24 2022] POST /identity/v3/users => generated 299 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:24.412766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5834095-3726-41c5-a357-f7902bde767e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:24.426070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5834095-3726-41c5-a357-f7902bde767e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:24.436198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c9676-012c-4ad9-8f5f-60a8df816325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:24.436959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c9676-012c-4ad9-8f5f-60a8df816325 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:24.437234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c9676-012c-4ad9-8f5f-60a8df816325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:24.437694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c9676-012c-4ad9-8f5f-60a8df816325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:24.438403 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c9676-012c-4ad9-8f5f-60a8df816325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:24.440068 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c9676-012c-4ad9-8f5f-60a8df816325 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:24.458304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1025/2058] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:24.462368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c29a83-2bf3-495d-9a5c-851e989ea7ed admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:24.478312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c29a83-2bf3-495d-9a5c-851e989ea7ed admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:24.487024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af4f7c-e73a-4458-9b8f-bea5e9f7cf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:24.488269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af4f7c-e73a-4458-9b8f-bea5e9f7cf1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:24.488591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af4f7c-e73a-4458-9b8f-bea5e9f7cf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:24.488868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af4f7c-e73a-4458-9b8f-bea5e9f7cf1e admin admin] PATH_INFO: `/v3/projects/8125bc49eda643a282501e1ada9fe4f9/users/0c4402e367804da6a1ce5b977b52b760/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:24.519978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af4f7c-e73a-4458-9b8f-bea5e9f7cf1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8125bc49eda643a282501e1ada9fe4f9, user_id=0c4402e367804da6a1ce5b977b52b760, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:24.523733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af4f7c-e73a-4458-9b8f-bea5e9f7cf1e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:24.635128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1034/2059] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:24 2022] PUT /identity/v3/projects/8125bc49eda643a282501e1ada9fe4f9/users/0c4402e367804da6a1ce5b977b52b760/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:24.640446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c9676-012c-4ad9-8f5f-60a8df816325 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:24.661851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c9676-012c-4ad9-8f5f-60a8df816325 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:24.729724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:24.771971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a651f-0803-40c2-96fb-3a30c053e6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:24.774157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a651f-0803-40c2-96fb-3a30c053e6ee admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:24.774674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a651f-0803-40c2-96fb-3a30c053e6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:24.775145 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a651f-0803-40c2-96fb-3a30c053e6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:24.776229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11a651f-0803-40c2-96fb-3a30c053e6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:24.778506 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11a651f-0803-40c2-96fb-3a30c053e6ee admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:24.797788 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1026/2060] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:24.802102 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af4f7c-e73a-4458-9b8f-bea5e9f7cf1e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:24.814052 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af4f7c-e73a-4458-9b8f-bea5e9f7cf1e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:24.819809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8e713-2341-41d4-a288-6e662baf4885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:24.820891 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8e713-2341-41d4-a288-6e662baf4885 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:24.821206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8e713-2341-41d4-a288-6e662baf4885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:24.821794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8e713-2341-41d4-a288-6e662baf4885 admin admin] PATH_INFO: `/v3/projects/8125bc49eda643a282501e1ada9fe4f9/users/0c4402e367804da6a1ce5b977b52b760/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:24.824269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8e713-2341-41d4-a288-6e662baf4885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8125bc49eda643a282501e1ada9fe4f9, user_id=0c4402e367804da6a1ce5b977b52b760, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:24.827066 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8e713-2341-41d4-a288-6e662baf4885 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:24.853415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1035/2061] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:24 2022] PUT /identity/v3/projects/8125bc49eda643a282501e1ada9fe4f9/users/0c4402e367804da6a1ce5b977b52b760/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:31.786740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a651f-0803-40c2-96fb-3a30c053e6ee admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:31.790583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e11a651f-0803-40c2-96fb-3a30c053e6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:31.822952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a651f-0803-40c2-96fb-3a30c053e6ee admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:31.967642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb4b95-60d1-4c3d-a581-88285ca32953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:31.968462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb4b95-60d1-4c3d-a581-88285ca32953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:31.970204 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb4b95-60d1-4c3d-a581-88285ca32953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:31.971357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb4b95-60d1-4c3d-a581-88285ca32953 admin admin] PATH_INFO: `/v3/users/00407384465c41848ad28a15729a95d6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:31.977099 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb4b95-60d1-4c3d-a581-88285ca32953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00407384465c41848ad28a15729a95d6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:31.979268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb4b95-60d1-4c3d-a581-88285ca32953 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:32.308544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-00fb4b95-60d1-4c3d-a581-88285ca32953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00407384465c41848ad28a15729a95d6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:32.330530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-00fb4b95-60d1-4c3d-a581-88285ca32953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00407384465c41848ad28a15729a95d6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:32.422251 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-00fb4b95-60d1-4c3d-a581-88285ca32953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00407384465c41848ad28a15729a95d6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:32.423331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-00fb4b95-60d1-4c3d-a581-88285ca32953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00407384465c41848ad28a15729a95d6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:32.436555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-00fb4b95-60d1-4c3d-a581-88285ca32953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00407384465c41848ad28a15729a95d6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:32.448903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1027/2062] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:31 2022] DELETE /identity/v3/users/00407384465c41848ad28a15729a95d6 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:33.102679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8e713-2341-41d4-a288-6e662baf4885 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:33.104781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e6b8e713-2341-41d4-a288-6e662baf4885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:33.139780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8e713-2341-41d4-a288-6e662baf4885 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:33.404027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f347ee-51a5-4856-8f44-493689e04a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:33.404897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f347ee-51a5-4856-8f44-493689e04a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:33.405284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f347ee-51a5-4856-8f44-493689e04a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:33.406125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f347ee-51a5-4856-8f44-493689e04a41 admin admin] PATH_INFO: `/v3/projects/7df041c538da419d8576d746b392c4db` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:33.408672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f347ee-51a5-4856-8f44-493689e04a41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df041c538da419d8576d746b392c4db)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:33.412416 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f347ee-51a5-4856-8f44-493689e04a41 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:33.571194 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:33.595978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-49f347ee-51a5-4856-8f44-493689e04a41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df041c538da419d8576d746b392c4db', 'request_id': 'req-49f347ee-51a5-4856-8f44-493689e04a41'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:33.727619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-49f347ee-51a5-4856-8f44-493689e04a41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df041c538da419d8576d746b392c4db', 'request_id': 'req-49f347ee-51a5-4856-8f44-493689e04a41'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:33.737924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-49f347ee-51a5-4856-8f44-493689e04a41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df041c538da419d8576d746b392c4db', 'request_id': 'req-49f347ee-51a5-4856-8f44-493689e04a41'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:33.749605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-49f347ee-51a5-4856-8f44-493689e04a41 admin admin] The token cache is being invalidate because project 7df041c538da419d8576d746b392c4db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:33.749913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-49f347ee-51a5-4856-8f44-493689e04a41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:33.750592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-49f347ee-51a5-4856-8f44-493689e04a41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:33.752125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1036/2063] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:33 2022] DELETE /identity/v3/projects/7df041c538da419d8576d746b392c4db => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:33.758045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bdf00-1c0d-4cb9-a874-8679049d771b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:33.758763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bdf00-1c0d-4cb9-a874-8679049d771b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:33.758763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bdf00-1c0d-4cb9-a874-8679049d771b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:33.790359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5c7bdf00-1c0d-4cb9-a874-8679049d771b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:33.791666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5c7bdf00-1c0d-4cb9-a874-8679049d771b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:33.848094 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:33.874225 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1028/2064] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:33.883658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b643fc-181c-4c83-85d7-5cc8cf2d8054 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:33.883658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b643fc-181c-4c83-85d7-5cc8cf2d8054 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:33.883658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b643fc-181c-4c83-85d7-5cc8cf2d8054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:33.903734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-19b643fc-181c-4c83-85d7-5cc8cf2d8054 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:33.905004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-19b643fc-181c-4c83-85d7-5cc8cf2d8054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:33.915462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1037/2065] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:33.919920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bdf00-1c0d-4cb9-a874-8679049d771b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:33.931208 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bdf00-1c0d-4cb9-a874-8679049d771b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:33.941593 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80aeab3-b25d-48b3-9612-33b6617efacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:33.942132 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aeab3-b25d-48b3-9612-33b6617efacb admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:33.942433 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aeab3-b25d-48b3-9612-33b6617efacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:33.942746 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aeab3-b25d-48b3-9612-33b6617efacb admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:33.943404 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80aeab3-b25d-48b3-9612-33b6617efacb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:48:33.943704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80aeab3-b25d-48b3-9612-33b6617efacb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:33.945972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80aeab3-b25d-48b3-9612-33b6617efacb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:33.972389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1029/2066] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:48:33.977436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b643fc-181c-4c83-85d7-5cc8cf2d8054 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:33.993764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b643fc-181c-4c83-85d7-5cc8cf2d8054 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:34.004537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9a051-b7aa-49ce-a150-9b5fc776b762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:34.005740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9a051-b7aa-49ce-a150-9b5fc776b762 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:34.005740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9a051-b7aa-49ce-a150-9b5fc776b762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:34.005740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9a051-b7aa-49ce-a150-9b5fc776b762 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:34.006396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9a051-b7aa-49ce-a150-9b5fc776b762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:34.008380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9a051-b7aa-49ce-a150-9b5fc776b762 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:34.132169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1038/2067] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:33 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:34.140103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80aeab3-b25d-48b3-9612-33b6617efacb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:34.154250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80aeab3-b25d-48b3-9612-33b6617efacb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:34.243125 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7c11b-903c-4af1-96f4-24ffa56166b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:34.243752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7c11b-903c-4af1-96f4-24ffa56166b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:34.244034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7c11b-903c-4af1-96f4-24ffa56166b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:34.244305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7c11b-903c-4af1-96f4-24ffa56166b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:34.244944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7c11b-903c-4af1-96f4-24ffa56166b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:34.247014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7c11b-903c-4af1-96f4-24ffa56166b4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:34.392793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1030/2068] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:34 2022] POST /identity/v3/users => generated 287 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:34.397771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9a051-b7aa-49ce-a150-9b5fc776b762 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:34.436608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9a051-b7aa-49ce-a150-9b5fc776b762 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:34.445317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcb84b-4ccf-4fa2-b4d2-7bd0ef598066 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:34.446887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcb84b-4ccf-4fa2-b4d2-7bd0ef598066 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:34.447314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcb84b-4ccf-4fa2-b4d2-7bd0ef598066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:34.469166 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4757153-be79-4831-ae64-95c6d533e3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:34.469796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4757153-be79-4831-ae64-95c6d533e3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:34.470070 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4757153-be79-4831-ae64-95c6d533e3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:34.470396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4757153-be79-4831-ae64-95c6d533e3c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:34.471059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4757153-be79-4831-ae64-95c6d533e3c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:34.472701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4757153-be79-4831-ae64-95c6d533e3c4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:34.496801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1039/2069] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:48:34.497867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4757153-be79-4831-ae64-95c6d533e3c4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:34.513649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-24dcb84b-4ccf-4fa2-b4d2-7bd0ef598066 None None] MFA Rules not processed for user `0c4402e367804da6a1ce5b977b52b760`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:34.514956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-24dcb84b-4ccf-4fa2-b4d2-7bd0ef598066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:34.520153 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4757153-be79-4831-ae64-95c6d533e3c4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:34.533444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356c53a-823c-47e3-a94d-b632d999ce07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:34.533444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c53a-823c-47e3-a94d-b632d999ce07 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:34.533444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c53a-823c-47e3-a94d-b632d999ce07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:34.533444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c53a-823c-47e3-a94d-b632d999ce07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:34.533885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356c53a-823c-47e3-a94d-b632d999ce07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:34.534216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356c53a-823c-47e3-a94d-b632d999ce07 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:34.545725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1040/2070] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:34.550426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356c53a-823c-47e3-a94d-b632d999ce07 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:34.562964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356c53a-823c-47e3-a94d-b632d999ce07 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:34.568463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc572c54-559c-4c91-90b0-3a13873862ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:34.578535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc572c54-559c-4c91-90b0-3a13873862ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:34.578535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc572c54-559c-4c91-90b0-3a13873862ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:34.578535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc572c54-559c-4c91-90b0-3a13873862ff admin admin] PATH_INFO: `/v3/projects/b17d6b7dccbb4c0ab1856cccbd71a376/users/473f7b876da44f2e8fc48ea515a5b1e3/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:34.598422 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc572c54-559c-4c91-90b0-3a13873862ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b17d6b7dccbb4c0ab1856cccbd71a376, user_id=473f7b876da44f2e8fc48ea515a5b1e3, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:34.602202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc572c54-559c-4c91-90b0-3a13873862ff admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:34.611911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1031/2071] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:34.629398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcb84b-4ccf-4fa2-b4d2-7bd0ef598066 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:34.635843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-24dcb84b-4ccf-4fa2-b4d2-7bd0ef598066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:34.745075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1041/2072] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:34 2022] PUT /identity/v3/projects/b17d6b7dccbb4c0ab1856cccbd71a376/users/473f7b876da44f2e8fc48ea515a5b1e3/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:34.759097 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71209be-27ec-4cca-a71b-1a2e749936ac None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:34.759532 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71209be-27ec-4cca-a71b-1a2e749936ac None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:34.759809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71209be-27ec-4cca-a71b-1a2e749936ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:34.786326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcb84b-4ccf-4fa2-b4d2-7bd0ef598066 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:34.807749 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f71209be-27ec-4cca-a71b-1a2e749936ac None None] MFA Rules not processed for user `473f7b876da44f2e8fc48ea515a5b1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:34.809279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f71209be-27ec-4cca-a71b-1a2e749936ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:34.878380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903100c-8195-48f8-9ccf-380995978e96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b86978df61245ae88fead154193fcd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:34.879719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903100c-8195-48f8-9ccf-380995978e96 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:34.880167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903100c-8195-48f8-9ccf-380995978e96 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:34.880653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903100c-8195-48f8-9ccf-380995978e96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:34.883774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c903100c-8195-48f8-9ccf-380995978e96 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:34.934331 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1042/2073] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:34.935729 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71209be-27ec-4cca-a71b-1a2e749936ac None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:34.939041 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f71209be-27ec-4cca-a71b-1a2e749936ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:34.977994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903100c-8195-48f8-9ccf-380995978e96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:34.977994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903100c-8195-48f8-9ccf-380995978e96 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:34.996590 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1032/2074] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:34.997639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903100c-8195-48f8-9ccf-380995978e96 service glance] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:35.012896 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:35.037295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71209be-27ec-4cca-a71b-1a2e749936ac None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:35.077128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438ef47-45cd-416e-8e83-66776897bd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:35.078571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438ef47-45cd-416e-8e83-66776897bd4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:35.078571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438ef47-45cd-416e-8e83-66776897bd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:35.080706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438ef47-45cd-416e-8e83-66776897bd4b admin admin] PATH_INFO: `/v3/users/157cc3f324004773a9d3f5d88cc955ed` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:35.080706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438ef47-45cd-416e-8e83-66776897bd4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=157cc3f324004773a9d3f5d88cc955ed)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:35.086064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438ef47-45cd-416e-8e83-66776897bd4b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:35.125729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903100c-8195-48f8-9ccf-380995978e96 service glance] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:35.133791 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d593a-a051-4cac-9374-15d35db08a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:35.134871 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d593a-a051-4cac-9374-15d35db08a77 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:35.135327 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d593a-a051-4cac-9374-15d35db08a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:35.136075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d593a-a051-4cac-9374-15d35db08a77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:35.137297 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d593a-a051-4cac-9374-15d35db08a77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:35.141354 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d593a-a051-4cac-9374-15d35db08a77 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:35.265027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1033/2075] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:34 2022] POST /identity/v3/projects => generated 371 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:35.269414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d593a-a051-4cac-9374-15d35db08a77 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:35.355148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0438ef47-45cd-416e-8e83-66776897bd4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '157cc3f324004773a9d3f5d88cc955ed'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:35.356005 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0438ef47-45cd-416e-8e83-66776897bd4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '157cc3f324004773a9d3f5d88cc955ed'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:35.359899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d593a-a051-4cac-9374-15d35db08a77 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:35.364915 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106cd3b-0ed0-410a-be9f-e056a2fafd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:35.366212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106cd3b-0ed0-410a-be9f-e056a2fafd74 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:35.366499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106cd3b-0ed0-410a-be9f-e056a2fafd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:35.366777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106cd3b-0ed0-410a-be9f-e056a2fafd74 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:35.367456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106cd3b-0ed0-410a-be9f-e056a2fafd74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:35.370534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106cd3b-0ed0-410a-be9f-e056a2fafd74 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:35.434518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0438ef47-45cd-416e-8e83-66776897bd4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '157cc3f324004773a9d3f5d88cc955ed'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:35.446606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0438ef47-45cd-416e-8e83-66776897bd4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '157cc3f324004773a9d3f5d88cc955ed'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:35.460291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0438ef47-45cd-416e-8e83-66776897bd4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '157cc3f324004773a9d3f5d88cc955ed'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:35.467544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1043/2076] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:34 2022] DELETE /identity/v3/users/157cc3f324004773a9d3f5d88cc955ed => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:35.468967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924fc00-ff91-4e36-8b65-133f28970b4d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:35.475293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924fc00-ff91-4e36-8b65-133f28970b4d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:35.475293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924fc00-ff91-4e36-8b65-133f28970b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:35.498312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a924fc00-ff91-4e36-8b65-133f28970b4d None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:35.498312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a924fc00-ff91-4e36-8b65-133f28970b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:35.500742 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:35.520009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1034/2077] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:35 2022] POST /identity/v3/users => generated 292 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:48:35.524031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db7393-f26a-4efc-9972-4cd6c3cedbf2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:35.524031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db7393-f26a-4efc-9972-4cd6c3cedbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:35.524031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db7393-f26a-4efc-9972-4cd6c3cedbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:35.550318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-18db7393-f26a-4efc-9972-4cd6c3cedbf2 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:35.551718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-18db7393-f26a-4efc-9972-4cd6c3cedbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:35.579968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1044/2078] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:48:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:35.581717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924fc00-ff91-4e36-8b65-133f28970b4d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:35.583977 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a924fc00-ff91-4e36-8b65-133f28970b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:35.649377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1035/2079] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:35.650308 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db7393-f26a-4efc-9972-4cd6c3cedbf2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:35.652130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-18db7393-f26a-4efc-9972-4cd6c3cedbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:35.707348 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924fc00-ff91-4e36-8b65-133f28970b4d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:35.713760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e7925-badc-4a0e-8d44-ddc9d75c76be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:35.714600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e7925-badc-4a0e-8d44-ddc9d75c76be admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:35.714882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e7925-badc-4a0e-8d44-ddc9d75c76be admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:35.715438 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e7925-badc-4a0e-8d44-ddc9d75c76be admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:35.715851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e7925-badc-4a0e-8d44-ddc9d75c76be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:35.717828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e7925-badc-4a0e-8d44-ddc9d75c76be admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:35.728838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1045/2080] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:35.729578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e7925-badc-4a0e-8d44-ddc9d75c76be admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:35.734507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-034e7925-badc-4a0e-8d44-ddc9d75c76be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:35.739921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db7393-f26a-4efc-9972-4cd6c3cedbf2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:35.747621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4281a5-d894-4aa0-a732-07e814b68d34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:35.748686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4281a5-d894-4aa0-a732-07e814b68d34 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:35.749025 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4281a5-d894-4aa0-a732-07e814b68d34 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:35.749298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4281a5-d894-4aa0-a732-07e814b68d34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:35.762083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4281a5-d894-4aa0-a732-07e814b68d34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:35.763846 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4281a5-d894-4aa0-a732-07e814b68d34 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:35.776886 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1036/2081] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:48:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:48:35.778049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4281a5-d894-4aa0-a732-07e814b68d34 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:35.794182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4281a5-d894-4aa0-a732-07e814b68d34 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:35.800630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb563f-b0ef-407e-8641-c7d4dd122e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:35.801361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb563f-b0ef-407e-8641-c7d4dd122e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:35.801750 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb563f-b0ef-407e-8641-c7d4dd122e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:35.802068 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb563f-b0ef-407e-8641-c7d4dd122e16 admin admin] PATH_INFO: `/v3/projects/9c224d9a12b847bdb07872502aca5ad8/users/0d577d8d30b6448a8a67714ac6ce3cf7/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:35.825657 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb563f-b0ef-407e-8641-c7d4dd122e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c224d9a12b847bdb07872502aca5ad8, user_id=0d577d8d30b6448a8a67714ac6ce3cf7, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:35.831725 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb563f-b0ef-407e-8641-c7d4dd122e16 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:35.840641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e7925-badc-4a0e-8d44-ddc9d75c76be admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:35.849083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d537226-bad5-44f9-9e46-fa3e1c123c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:35.849606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d537226-bad5-44f9-9e46-fa3e1c123c8b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:35.850582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d537226-bad5-44f9-9e46-fa3e1c123c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:35.850983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d537226-bad5-44f9-9e46-fa3e1c123c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:35.879959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d537226-bad5-44f9-9e46-fa3e1c123c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:35.881482 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d537226-bad5-44f9-9e46-fa3e1c123c8b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:35.892716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1046/2082] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:35.963145 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1037/2083] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:35 2022] PUT /identity/v3/projects/9c224d9a12b847bdb07872502aca5ad8/users/0d577d8d30b6448a8a67714ac6ce3cf7/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:35.968624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d537226-bad5-44f9-9e46-fa3e1c123c8b service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:35.982806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d537226-bad5-44f9-9e46-fa3e1c123c8b service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:36.131066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:36.150733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:36.164559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c7feb-61ae-413c-a96a-c2401eb352f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:36.165551 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c7feb-61ae-413c-a96a-c2401eb352f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:36.165827 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c7feb-61ae-413c-a96a-c2401eb352f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:36.166096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c7feb-61ae-413c-a96a-c2401eb352f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:36.166815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c7feb-61ae-413c-a96a-c2401eb352f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:36.168435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c7feb-61ae-413c-a96a-c2401eb352f4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:36.202212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1047/2084] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:48:36.206973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb563f-b0ef-407e-8641-c7d4dd122e16 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:36.221016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb563f-b0ef-407e-8641-c7d4dd122e16 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:36.228835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716c475-5d97-4338-9bc3-c1520a933208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:36.229672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716c475-5d97-4338-9bc3-c1520a933208 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:36.230039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716c475-5d97-4338-9bc3-c1520a933208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:36.230501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716c475-5d97-4338-9bc3-c1520a933208 admin admin] PATH_INFO: `/v3/projects/9c224d9a12b847bdb07872502aca5ad8/users/0d577d8d30b6448a8a67714ac6ce3cf7/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:36.247847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716c475-5d97-4338-9bc3-c1520a933208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c224d9a12b847bdb07872502aca5ad8, user_id=0d577d8d30b6448a8a67714ac6ce3cf7, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:36.251173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716c475-5d97-4338-9bc3-c1520a933208 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:36.271232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1038/2085] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:36 2022] PUT /identity/v3/projects/9c224d9a12b847bdb07872502aca5ad8/users/0d577d8d30b6448a8a67714ac6ce3cf7/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:36.285629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67e54f-bee9-4d83-b543-e1bf5b7960fc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:36.286330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67e54f-bee9-4d83-b543-e1bf5b7960fc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:36.286757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67e54f-bee9-4d83-b543-e1bf5b7960fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:36.348315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7a67e54f-bee9-4d83-b543-e1bf5b7960fc None None] MFA Rules not processed for user `0d577d8d30b6448a8a67714ac6ce3cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:36.352321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7a67e54f-bee9-4d83-b543-e1bf5b7960fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:36.545647 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1048/2086] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:36.563079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716c475-5d97-4338-9bc3-c1520a933208 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:36.566010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b716c475-5d97-4338-9bc3-c1520a933208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:36.599731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:36.660173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716c475-5d97-4338-9bc3-c1520a933208 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:36.695411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82b9d6-0ef0-4ab3-a0ea-911bd224e7d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:36.696058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82b9d6-0ef0-4ab3-a0ea-911bd224e7d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:36.696336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82b9d6-0ef0-4ab3-a0ea-911bd224e7d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:36.696609 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82b9d6-0ef0-4ab3-a0ea-911bd224e7d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:36.714437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67e54f-bee9-4d83-b543-e1bf5b7960fc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:36.716074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7a67e54f-bee9-4d83-b543-e1bf5b7960fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:36.759602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82b9d6-0ef0-4ab3-a0ea-911bd224e7d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:36.762953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82b9d6-0ef0-4ab3-a0ea-911bd224e7d6 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:36.800910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1039/2087] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:36.865631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:36.957584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67e54f-bee9-4d83-b543-e1bf5b7960fc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:37.015963 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:37.017254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:37.017757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:37.019202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] PATH_INFO: `/v3/projects/0321961b0c044d9cb899ac92d1771422` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:37.021886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0321961b0c044d9cb899ac92d1771422)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:37.050606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:37.402931 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0321961b0c044d9cb899ac92d1771422', 'request_id': 'req-931dcc57-8bc6-41cc-8c4c-8430d1998abf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:37.495935 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0321961b0c044d9cb899ac92d1771422', 'request_id': 'req-931dcc57-8bc6-41cc-8c4c-8430d1998abf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:37.524808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0321961b0c044d9cb899ac92d1771422', 'request_id': 'req-931dcc57-8bc6-41cc-8c4c-8430d1998abf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:37.563606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] The token cache is being invalidate because project 0321961b0c044d9cb899ac92d1771422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:37.563606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:37.565826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:37.578870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1049/2088] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:36 2022] DELETE /identity/v3/projects/0321961b0c044d9cb899ac92d1771422 => generated 0 bytes in 865 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:37.591848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82b9d6-0ef0-4ab3-a0ea-911bd224e7d6 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:37.624877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82b9d6-0ef0-4ab3-a0ea-911bd224e7d6 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:37.794685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:37.797230 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:37.797511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:37.797781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] PATH_INFO: `/v3/users/9135da1f04aa4000982d44c15e73e46c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:37.803439 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] RBAC: Authorizing `identity:delete_user(user_id=9135da1f04aa4000982d44c15e73e46c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:37.811049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:38.129883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9135da1f04aa4000982d44c15e73e46c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:38.197327 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9135da1f04aa4000982d44c15e73e46c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:38.337253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9135da1f04aa4000982d44c15e73e46c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:38.338774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9135da1f04aa4000982d44c15e73e46c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:38.353012 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9135da1f04aa4000982d44c15e73e46c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:38.371970 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1040/2089] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:37 2022] DELETE /identity/v3/users/9135da1f04aa4000982d44c15e73e46c => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:38.907659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:38.911628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:38.968591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dcc57-8bc6-41cc-8c4c-8430d1998abf admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:39.105698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a7406-4955-46ee-a4ef-2efca1e679f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:39.106942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7406-4955-46ee-a4ef-2efca1e679f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:39.107761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7406-4955-46ee-a4ef-2efca1e679f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:39.108317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7406-4955-46ee-a4ef-2efca1e679f2 admin admin] PATH_INFO: `/v3/users/473f7b876da44f2e8fc48ea515a5b1e3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:39.111800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a7406-4955-46ee-a4ef-2efca1e679f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=473f7b876da44f2e8fc48ea515a5b1e3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:39.118251 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a7406-4955-46ee-a4ef-2efca1e679f2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:39.418027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:39.420688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:39.489283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-057a7406-4955-46ee-a4ef-2efca1e679f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '473f7b876da44f2e8fc48ea515a5b1e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:39.490307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-057a7406-4955-46ee-a4ef-2efca1e679f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '473f7b876da44f2e8fc48ea515a5b1e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:39.493023 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60980beb-fbb5-4a09-ae5d-bbc27141dfea admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:39.500634 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313a8f1-789d-4066-9296-d05f64a4853c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:39.501654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313a8f1-789d-4066-9296-d05f64a4853c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:39.502381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313a8f1-789d-4066-9296-d05f64a4853c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:39.502901 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313a8f1-789d-4066-9296-d05f64a4853c admin admin] PATH_INFO: `/v3/projects/8dd0fb04a9ce4aedbf7218cc893df256` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:39.504664 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313a8f1-789d-4066-9296-d05f64a4853c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd0fb04a9ce4aedbf7218cc893df256)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:39.507986 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313a8f1-789d-4066-9296-d05f64a4853c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:39.588932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-057a7406-4955-46ee-a4ef-2efca1e679f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '473f7b876da44f2e8fc48ea515a5b1e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:39.599431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:39.603746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-057a7406-4955-46ee-a4ef-2efca1e679f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '473f7b876da44f2e8fc48ea515a5b1e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:39.623375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-057a7406-4955-46ee-a4ef-2efca1e679f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '473f7b876da44f2e8fc48ea515a5b1e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:39.636444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1050/2090] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:38 2022] DELETE /identity/v3/users/473f7b876da44f2e8fc48ea515a5b1e3 => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:39.645886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743f06e-a2d4-44ee-955e-20af475e1629 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:39.646460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743f06e-a2d4-44ee-955e-20af475e1629 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:39.648230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743f06e-a2d4-44ee-955e-20af475e1629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:39.688410 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:39.731928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6743f06e-a2d4-44ee-955e-20af475e1629 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:39.734103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6743f06e-a2d4-44ee-955e-20af475e1629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:39.746115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1313a8f1-789d-4066-9296-d05f64a4853c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd0fb04a9ce4aedbf7218cc893df256', 'request_id': 'req-1313a8f1-789d-4066-9296-d05f64a4853c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:39.879643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1051/2091] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:39.888656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743f06e-a2d4-44ee-955e-20af475e1629 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:39.890104 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6743f06e-a2d4-44ee-955e-20af475e1629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:39.904078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1313a8f1-789d-4066-9296-d05f64a4853c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd0fb04a9ce4aedbf7218cc893df256', 'request_id': 'req-1313a8f1-789d-4066-9296-d05f64a4853c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:39.914644 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1313a8f1-789d-4066-9296-d05f64a4853c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd0fb04a9ce4aedbf7218cc893df256', 'request_id': 'req-1313a8f1-789d-4066-9296-d05f64a4853c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:39.945280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1313a8f1-789d-4066-9296-d05f64a4853c admin admin] The token cache is being invalidate because project 8dd0fb04a9ce4aedbf7218cc893df256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:39.946030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1313a8f1-789d-4066-9296-d05f64a4853c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:39.947246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1313a8f1-789d-4066-9296-d05f64a4853c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:39.954753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1041/2092] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:39 2022] DELETE /identity/v3/projects/8dd0fb04a9ce4aedbf7218cc893df256 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:39.957182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7c806-1090-4e65-bb9c-bce1b4e03778 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:39.957879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7c806-1090-4e65-bb9c-bce1b4e03778 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:39.958478 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7c806-1090-4e65-bb9c-bce1b4e03778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:40.011919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dca7c806-1090-4e65-bb9c-bce1b4e03778 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:40.019674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dca7c806-1090-4e65-bb9c-bce1b4e03778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:40.039462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1042/2093] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:40.047648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722513e-bc58-401e-8258-3334933296ba None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:40.047964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722513e-bc58-401e-8258-3334933296ba None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:40.048240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722513e-bc58-401e-8258-3334933296ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:40.077028 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743f06e-a2d4-44ee-955e-20af475e1629 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:40.087376 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce909a4d-e116-4618-855e-3d514d224fbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:40.088003 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce909a4d-e116-4618-855e-3d514d224fbe service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:40.088657 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce909a4d-e116-4618-855e-3d514d224fbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:40.089080 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce909a4d-e116-4618-855e-3d514d224fbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:40.092304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2722513e-bc58-401e-8258-3334933296ba None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:40.093254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ce909a4d-e116-4618-855e-3d514d224fbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:40.096743 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2722513e-bc58-401e-8258-3334933296ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:40.117422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1043/2094] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:40.118689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722513e-bc58-401e-8258-3334933296ba None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:40.130275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce909a4d-e116-4618-855e-3d514d224fbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:40.135811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722513e-bc58-401e-8258-3334933296ba None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:40.140004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce909a4d-e116-4618-855e-3d514d224fbe service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:40.159992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1052/2095] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:48:40.161933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2b5f4-54c5-49b9-9893-896e63995ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:40.163086 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2b5f4-54c5-49b9-9893-896e63995ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:40.166005 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2b5f4-54c5-49b9-9893-896e63995ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:40.166521 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2b5f4-54c5-49b9-9893-896e63995ae5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:40.167781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2b5f4-54c5-49b9-9893-896e63995ae5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:48:40.168434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2b5f4-54c5-49b9-9893-896e63995ae5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:40.171670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2b5f4-54c5-49b9-9893-896e63995ae5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:40.196803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1044/2096] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:48:40.197061 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce909a4d-e116-4618-855e-3d514d224fbe service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:40.210681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce909a4d-e116-4618-855e-3d514d224fbe service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:40.216303 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958df705-5d06-45f7-b32b-10b13b2fde9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:40.237704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958df705-5d06-45f7-b32b-10b13b2fde9e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:40.238015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958df705-5d06-45f7-b32b-10b13b2fde9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:40.238554 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958df705-5d06-45f7-b32b-10b13b2fde9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:40.239255 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958df705-5d06-45f7-b32b-10b13b2fde9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:40.243213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958df705-5d06-45f7-b32b-10b13b2fde9e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:40.358032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1053/2097] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:40 2022] POST /identity/v3/projects => generated 401 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:40.361962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2b5f4-54c5-49b9-9893-896e63995ae5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:40.378577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2b5f4-54c5-49b9-9893-896e63995ae5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:40.539185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d6cf5-0b23-4d47-8404-d2e35ebf9bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:40.541589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d6cf5-0b23-4d47-8404-d2e35ebf9bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:40.543168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d6cf5-0b23-4d47-8404-d2e35ebf9bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:40.543917 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d6cf5-0b23-4d47-8404-d2e35ebf9bd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:40.544851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d6cf5-0b23-4d47-8404-d2e35ebf9bd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:40.551833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d6cf5-0b23-4d47-8404-d2e35ebf9bd2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:40.729583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1045/2098] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:40 2022] POST /identity/v3/users => generated 301 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:40.730662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958df705-5d06-45f7-b32b-10b13b2fde9e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:40.798443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958df705-5d06-45f7-b32b-10b13b2fde9e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:40.838325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fb858-b806-4dbc-bd3d-cf6f8d7acac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:40.839732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fb858-b806-4dbc-bd3d-cf6f8d7acac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:40.840380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fb858-b806-4dbc-bd3d-cf6f8d7acac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:40.841065 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fb858-b806-4dbc-bd3d-cf6f8d7acac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:40.842538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fb858-b806-4dbc-bd3d-cf6f8d7acac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:40.847205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fb858-b806-4dbc-bd3d-cf6f8d7acac4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:40.915047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1054/2099] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:40.920051 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d6cf5-0b23-4d47-8404-d2e35ebf9bd2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:40.981751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d6cf5-0b23-4d47-8404-d2e35ebf9bd2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:41.004582 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc864f-2e44-45f6-b771-5cb640490c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:41.008166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc864f-2e44-45f6-b771-5cb640490c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:41.008730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc864f-2e44-45f6-b771-5cb640490c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:41.009166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc864f-2e44-45f6-b771-5cb640490c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:41.011288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc864f-2e44-45f6-b771-5cb640490c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:41.018439 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc864f-2e44-45f6-b771-5cb640490c08 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:41.055305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1046/2100] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:41.058019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fb858-b806-4dbc-bd3d-cf6f8d7acac4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:41.105222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fb858-b806-4dbc-bd3d-cf6f8d7acac4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:41.121902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e890d83-5061-4412-b6dd-dc8be2a25256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:41.122603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e890d83-5061-4412-b6dd-dc8be2a25256 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:41.126908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e890d83-5061-4412-b6dd-dc8be2a25256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:41.126908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e890d83-5061-4412-b6dd-dc8be2a25256 admin admin] PATH_INFO: `/v3/projects/68504220ca0d40af8157b20db9e4ba86/users/59212cc1419848f189d8c582bce27ac2/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:41.129792 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc864f-2e44-45f6-b771-5cb640490c08 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:41.131791 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-73fc864f-2e44-45f6-b771-5cb640490c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:41.154801 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc864f-2e44-45f6-b771-5cb640490c08 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:41.167786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:41.168485 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:41.169074 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:41.169365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] PATH_INFO: `/v3/projects/b17d6b7dccbb4c0ab1856cccbd71a376` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:41.170976 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] RBAC: Authorizing `identity:delete_project(project_id=b17d6b7dccbb4c0ab1856cccbd71a376)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:41.172983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:41.176086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e890d83-5061-4412-b6dd-dc8be2a25256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68504220ca0d40af8157b20db9e4ba86, user_id=59212cc1419848f189d8c582bce27ac2, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:41.185297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e890d83-5061-4412-b6dd-dc8be2a25256 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:41.305716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1055/2101] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:41 2022] PUT /identity/v3/projects/68504220ca0d40af8157b20db9e4ba86/users/59212cc1419848f189d8c582bce27ac2/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:41.320076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf45db0-b46e-4dcc-ab6c-80f7f7fee396 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:41.320452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf45db0-b46e-4dcc-ab6c-80f7f7fee396 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:41.320828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf45db0-b46e-4dcc-ab6c-80f7f7fee396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:41.407980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b17d6b7dccbb4c0ab1856cccbd71a376', 'request_id': 'req-0f841ad5-dcdc-48ce-b188-a71d591dadca'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:41.421359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fbf45db0-b46e-4dcc-ab6c-80f7f7fee396 None None] MFA Rules not processed for user `59212cc1419848f189d8c582bce27ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:41.421359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fbf45db0-b46e-4dcc-ab6c-80f7f7fee396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:41.443967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:41.513930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1056/2102] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:41 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:41.519533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf45db0-b46e-4dcc-ab6c-80f7f7fee396 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:41.599295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf45db0-b46e-4dcc-ab6c-80f7f7fee396 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:41.604082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b22a0-8104-4863-a9e1-cdbf93c7c395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:41.605499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b22a0-8104-4863-a9e1-cdbf93c7c395 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:41.605784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b22a0-8104-4863-a9e1-cdbf93c7c395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:41.606044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b22a0-8104-4863-a9e1-cdbf93c7c395 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:41.606721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b22a0-8104-4863-a9e1-cdbf93c7c395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:41.609144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b22a0-8104-4863-a9e1-cdbf93c7c395 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:41.889727 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b17d6b7dccbb4c0ab1856cccbd71a376', 'request_id': 'req-0f841ad5-dcdc-48ce-b188-a71d591dadca'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:41.899698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:41.901311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b17d6b7dccbb4c0ab1856cccbd71a376', 'request_id': 'req-0f841ad5-dcdc-48ce-b188-a71d591dadca'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:41.915765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] The token cache is being invalidate because project b17d6b7dccbb4c0ab1856cccbd71a376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:41.916162 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:41.916923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:41.918856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1047/2103] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:41 2022] DELETE /identity/v3/projects/b17d6b7dccbb4c0ab1856cccbd71a376 => generated 0 bytes in 789 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:41.929339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:41.931292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:41.954558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1057/2104] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:41 2022] POST /identity/v3/projects => generated 399 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:41.957193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f841ad5-dcdc-48ce-b188-a71d591dadca admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:41.961653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b22a0-8104-4863-a9e1-cdbf93c7c395 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:41.964428 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2d3b22a0-8104-4863-a9e1-cdbf93c7c395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:42.143411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:42.144114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:42.144382 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:42.144832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] PATH_INFO: `/v3/users/0d577d8d30b6448a8a67714ac6ce3cf7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:42.147202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d577d8d30b6448a8a67714ac6ce3cf7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:42.149694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:42.208810 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b22a0-8104-4863-a9e1-cdbf93c7c395 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:42.222327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a14e5-ae2d-4f2d-ac22-a7c0a3217d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:42.223962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a14e5-ae2d-4f2d-ac22-a7c0a3217d9e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:42.225305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a14e5-ae2d-4f2d-ac22-a7c0a3217d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:42.225583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a14e5-ae2d-4f2d-ac22-a7c0a3217d9e admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:42.226837 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a14e5-ae2d-4f2d-ac22-a7c0a3217d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:42.233064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a14e5-ae2d-4f2d-ac22-a7c0a3217d9e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:42.365542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1058/2105] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:41 2022] POST /identity/v3/users => generated 306 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:42.370943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a14e5-ae2d-4f2d-ac22-a7c0a3217d9e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:42.389533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a14e5-ae2d-4f2d-ac22-a7c0a3217d9e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:42.395640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b32563-6db5-498c-8655-3dc5512a33f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:42.397271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b32563-6db5-498c-8655-3dc5512a33f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:42.397676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b32563-6db5-498c-8655-3dc5512a33f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:42.398079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b32563-6db5-498c-8655-3dc5512a33f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:42.398887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b32563-6db5-498c-8655-3dc5512a33f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:42.401411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b32563-6db5-498c-8655-3dc5512a33f0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:42.408522 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d577d8d30b6448a8a67714ac6ce3cf7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:42.414724 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1059/2106] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:48:42.419205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b32563-6db5-498c-8655-3dc5512a33f0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:42.433107 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d577d8d30b6448a8a67714ac6ce3cf7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:42.448742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b32563-6db5-498c-8655-3dc5512a33f0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:42.462369 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94a347-ab15-4ba8-a24e-9aa37a066c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:42.463051 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94a347-ab15-4ba8-a24e-9aa37a066c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:42.463327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94a347-ab15-4ba8-a24e-9aa37a066c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:42.463644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94a347-ab15-4ba8-a24e-9aa37a066c98 admin admin] PATH_INFO: `/v3/projects/516977cec4c942bdb5f8bad3037b0876/users/39e88e65c9bf4ab6a0d6ce4187beb0b5/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:42.489563 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94a347-ab15-4ba8-a24e-9aa37a066c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=516977cec4c942bdb5f8bad3037b0876, user_id=39e88e65c9bf4ab6a0d6ce4187beb0b5, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:42.493861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94a347-ab15-4ba8-a24e-9aa37a066c98 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:42.545594 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d577d8d30b6448a8a67714ac6ce3cf7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:42.546696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d577d8d30b6448a8a67714ac6ce3cf7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:42.558901 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d577d8d30b6448a8a67714ac6ce3cf7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:42.574315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1048/2107] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:41 2022] DELETE /identity/v3/users/0d577d8d30b6448a8a67714ac6ce3cf7 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:42.583173 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:42.677287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1060/2108] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:42 2022] PUT /identity/v3/projects/516977cec4c942bdb5f8bad3037b0876/users/39e88e65c9bf4ab6a0d6ce4187beb0b5/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:42.682075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:42.682983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:42.794968 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b09248-b2bc-40fa-bdf3-c929d487d190 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:42.837012 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:42.937015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c800b5e-20e0-4643-89c4-393f9a3afb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:42.938927 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c800b5e-20e0-4643-89c4-393f9a3afb90 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:42.939482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c800b5e-20e0-4643-89c4-393f9a3afb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:42.939946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c800b5e-20e0-4643-89c4-393f9a3afb90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:42.940647 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c800b5e-20e0-4643-89c4-393f9a3afb90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:42.942407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c800b5e-20e0-4643-89c4-393f9a3afb90 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:42.959669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1049/2109] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:48:42.969254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94a347-ab15-4ba8-a24e-9aa37a066c98 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:42.985245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94a347-ab15-4ba8-a24e-9aa37a066c98 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:42.991748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93e6f2-c14f-49c1-9273-f58bac07fdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:42.992611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93e6f2-c14f-49c1-9273-f58bac07fdb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:42.992743 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93e6f2-c14f-49c1-9273-f58bac07fdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:42.992995 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93e6f2-c14f-49c1-9273-f58bac07fdb8 admin admin] PATH_INFO: `/v3/projects/516977cec4c942bdb5f8bad3037b0876/users/39e88e65c9bf4ab6a0d6ce4187beb0b5/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:43.000917 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93e6f2-c14f-49c1-9273-f58bac07fdb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=516977cec4c942bdb5f8bad3037b0876, user_id=39e88e65c9bf4ab6a0d6ce4187beb0b5, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:43.003784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93e6f2-c14f-49c1-9273-f58bac07fdb8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:43.017469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1061/2110] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:42 2022] PUT /identity/v3/projects/516977cec4c942bdb5f8bad3037b0876/users/39e88e65c9bf4ab6a0d6ce4187beb0b5/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:43.031950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d756123-ccb9-47d9-b70d-179d98b6afc8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:43.031950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d756123-ccb9-47d9-b70d-179d98b6afc8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:43.032495 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d756123-ccb9-47d9-b70d-179d98b6afc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:43.093597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7d756123-ccb9-47d9-b70d-179d98b6afc8 None None] MFA Rules not processed for user `39e88e65c9bf4ab6a0d6ce4187beb0b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:43.094944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7d756123-ccb9-47d9-b70d-179d98b6afc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:43.349807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93e6f2-c14f-49c1-9273-f58bac07fdb8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:43.353417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9f93e6f2-c14f-49c1-9273-f58bac07fdb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:43.379026 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1050/2111] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:43 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:43.402106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d756123-ccb9-47d9-b70d-179d98b6afc8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:43.404784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7d756123-ccb9-47d9-b70d-179d98b6afc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:43.583309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d756123-ccb9-47d9-b70d-179d98b6afc8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:43.599218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72419b2d-8fa0-43c2-8914-04608a99d9bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:43.599218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72419b2d-8fa0-43c2-8914-04608a99d9bd service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:43.599218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72419b2d-8fa0-43c2-8914-04608a99d9bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:43.599218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72419b2d-8fa0-43c2-8914-04608a99d9bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:43.605147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93e6f2-c14f-49c1-9273-f58bac07fdb8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:43.611263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:43.613443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:43.614426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:43.614841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94 admin admin] PATH_INFO: `/v3/projects/9c224d9a12b847bdb07872502aca5ad8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:43.617078 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c224d9a12b847bdb07872502aca5ad8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:43.619790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:43.623100 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72419b2d-8fa0-43c2-8914-04608a99d9bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:43.624832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72419b2d-8fa0-43c2-8914-04608a99d9bd service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:43.642199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1051/2112] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:43 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:48:43.835775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c224d9a12b847bdb07872502aca5ad8', 'request_id': 'req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:43.932172 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72419b2d-8fa0-43c2-8914-04608a99d9bd service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:43.942053 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c224d9a12b847bdb07872502aca5ad8', 'request_id': 'req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:43.955077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c224d9a12b847bdb07872502aca5ad8', 'request_id': 'req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:43.955582 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72419b2d-8fa0-43c2-8914-04608a99d9bd service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:43.983924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94 admin admin] The token cache is being invalidate because project 9c224d9a12b847bdb07872502aca5ad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:43.984206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:43.985195 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0047421c-e9db-4e34-9ef8-2ec64aa0ab94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:43.986741 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1062/2113] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:43 2022] DELETE /identity/v3/projects/9c224d9a12b847bdb07872502aca5ad8 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:43.996906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4add4-f7a7-4e3a-ad28-86e50251c811 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:43.997731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4add4-f7a7-4e3a-ad28-86e50251c811 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:43.998418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4add4-f7a7-4e3a-ad28-86e50251c811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:44.074412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2ab4add4-f7a7-4e3a-ad28-86e50251c811 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:44.075845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:44.077855 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2ab4add4-f7a7-4e3a-ad28-86e50251c811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:44.099483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61b22e-453e-48b8-8760-5530a1db74d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:44.100256 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61b22e-453e-48b8-8760-5530a1db74d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:44.100710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61b22e-453e-48b8-8760-5530a1db74d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:44.100988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61b22e-453e-48b8-8760-5530a1db74d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:44.105703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6f61b22e-453e-48b8-8760-5530a1db74d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:44.216166 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1063/2114] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:44.223869 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd018be-0153-43fb-9996-5eb45f4ed619 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:44.224412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd018be-0153-43fb-9996-5eb45f4ed619 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:44.224412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd018be-0153-43fb-9996-5eb45f4ed619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:44.276201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61b22e-453e-48b8-8760-5530a1db74d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:44.278322 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61b22e-453e-48b8-8760-5530a1db74d0 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:44.283070 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6dd018be-0153-43fb-9996-5eb45f4ed619 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:44.284315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6dd018be-0153-43fb-9996-5eb45f4ed619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:44.304472 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1064/2116] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:44.304918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1052/2116] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:43 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:48:44.309590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd018be-0153-43fb-9996-5eb45f4ed619 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:44.316509 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:44.331938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd018be-0153-43fb-9996-5eb45f4ed619 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:44.337945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf35355-f865-4677-aa16-786d03655b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:44.338673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf35355-f865-4677-aa16-786d03655b07 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:44.338942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf35355-f865-4677-aa16-786d03655b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:44.339352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf35355-f865-4677-aa16-786d03655b07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:44.340343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf35355-f865-4677-aa16-786d03655b07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:48:44.340852 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf35355-f865-4677-aa16-786d03655b07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:44.342877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf35355-f865-4677-aa16-786d03655b07 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:44.380508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1065/2117] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:48:44.388015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61b22e-453e-48b8-8760-5530a1db74d0 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:44.414285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61b22e-453e-48b8-8760-5530a1db74d0 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:44.425732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b50b5e-0134-49f7-891b-39200c5822d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:44.427667 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b50b5e-0134-49f7-891b-39200c5822d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:44.430834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b50b5e-0134-49f7-891b-39200c5822d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:44.431434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b50b5e-0134-49f7-891b-39200c5822d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:44.435139 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b50b5e-0134-49f7-891b-39200c5822d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:44.439383 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b50b5e-0134-49f7-891b-39200c5822d8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:44.661225 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1053/2118] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:44 2022] POST /identity/v3/projects => generated 369 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:44.666121 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf35355-f865-4677-aa16-786d03655b07 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:44.686193 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf35355-f865-4677-aa16-786d03655b07 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:44.815761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93288f4-4dc4-4f84-b01e-ce096accbad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:44.816698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93288f4-4dc4-4f84-b01e-ce096accbad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:44.816827 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93288f4-4dc4-4f84-b01e-ce096accbad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:44.817066 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93288f4-4dc4-4f84-b01e-ce096accbad3 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:44.817691 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93288f4-4dc4-4f84-b01e-ce096accbad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:44.820668 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93288f4-4dc4-4f84-b01e-ce096accbad3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:44.953481 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1066/2119] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:44 2022] POST /identity/v3/users => generated 285 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:44.954980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b50b5e-0134-49f7-891b-39200c5822d8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:44.970708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b50b5e-0134-49f7-891b-39200c5822d8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:44.976927 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b33ba9-1596-4f3b-a7f7-fafaf0862aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:44.986330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b33ba9-1596-4f3b-a7f7-fafaf0862aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:44.988671 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b33ba9-1596-4f3b-a7f7-fafaf0862aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:44.988671 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b33ba9-1596-4f3b-a7f7-fafaf0862aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:44.988671 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b33ba9-1596-4f3b-a7f7-fafaf0862aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:44.989479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b33ba9-1596-4f3b-a7f7-fafaf0862aa9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:45.047544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1054/2120] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:45.049676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93288f4-4dc4-4f84-b01e-ce096accbad3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:45.079521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93288f4-4dc4-4f84-b01e-ce096accbad3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:45.095931 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39643ac1-4a39-40ab-9236-0566a9bcb9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:45.096611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39643ac1-4a39-40ab-9236-0566a9bcb9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:45.096968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39643ac1-4a39-40ab-9236-0566a9bcb9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:45.097304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39643ac1-4a39-40ab-9236-0566a9bcb9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:45.099136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39643ac1-4a39-40ab-9236-0566a9bcb9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:45.100953 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39643ac1-4a39-40ab-9236-0566a9bcb9a3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:45.124869 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1067/2121] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:45.128329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b33ba9-1596-4f3b-a7f7-fafaf0862aa9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:45.141010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b33ba9-1596-4f3b-a7f7-fafaf0862aa9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:45.149573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3961b-ceb6-44be-b184-0c69420d3466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:45.150367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3961b-ceb6-44be-b184-0c69420d3466 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:45.150723 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3961b-ceb6-44be-b184-0c69420d3466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:45.150994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3961b-ceb6-44be-b184-0c69420d3466 admin admin] PATH_INFO: `/v3/projects/668ee817c28d4bda8e2bd04b4783973c/users/ca4570b209464399b08c257d21ed1a55/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:45.172642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3961b-ceb6-44be-b184-0c69420d3466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=668ee817c28d4bda8e2bd04b4783973c, user_id=ca4570b209464399b08c257d21ed1a55, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:45.175413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3961b-ceb6-44be-b184-0c69420d3466 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:45.275688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1055/2122] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:45 2022] PUT /identity/v3/projects/668ee817c28d4bda8e2bd04b4783973c/users/ca4570b209464399b08c257d21ed1a55/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:45.290629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899b429-19a0-4e63-ab82-37a6dfe1e97e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:45.291778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899b429-19a0-4e63-ab82-37a6dfe1e97e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:45.291778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899b429-19a0-4e63-ab82-37a6dfe1e97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:45.357964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3899b429-19a0-4e63-ab82-37a6dfe1e97e None None] MFA Rules not processed for user `ca4570b209464399b08c257d21ed1a55`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:45.359256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3899b429-19a0-4e63-ab82-37a6dfe1e97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:45.507770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1068/2123] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:45.512387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3961b-ceb6-44be-b184-0c69420d3466 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:45.622809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3961b-ceb6-44be-b184-0c69420d3466 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:45.629927 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349ee30-dcb1-4c56-8132-45cac8a9afe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:45.632276 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349ee30-dcb1-4c56-8132-45cac8a9afe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:45.633133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349ee30-dcb1-4c56-8132-45cac8a9afe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:45.633864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349ee30-dcb1-4c56-8132-45cac8a9afe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:45.635415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349ee30-dcb1-4c56-8132-45cac8a9afe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:45.639704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349ee30-dcb1-4c56-8132-45cac8a9afe0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:45.789702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1056/2124] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:45 2022] POST /identity/v3/projects => generated 369 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:45.793693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899b429-19a0-4e63-ab82-37a6dfe1e97e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:45.811129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899b429-19a0-4e63-ab82-37a6dfe1e97e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:45.941758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e6e87-acb8-4791-b42d-acee567655db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:45.952199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e6e87-acb8-4791-b42d-acee567655db admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:45.952460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e6e87-acb8-4791-b42d-acee567655db admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:45.952742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e6e87-acb8-4791-b42d-acee567655db admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:45.953486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e6e87-acb8-4791-b42d-acee567655db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:45.955635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e6e87-acb8-4791-b42d-acee567655db admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:45.979131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:46.116119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349ee30-dcb1-4c56-8132-45cac8a9afe0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:46.118035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1349ee30-dcb1-4c56-8132-45cac8a9afe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:46.131562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349ee30-dcb1-4c56-8132-45cac8a9afe0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:46.140106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bfac3-8d8c-4b7c-98eb-7abb111a61a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:46.140106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bfac3-8d8c-4b7c-98eb-7abb111a61a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:46.140106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bfac3-8d8c-4b7c-98eb-7abb111a61a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:46.140106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bfac3-8d8c-4b7c-98eb-7abb111a61a7 admin admin] PATH_INFO: `/v3/users/863637528a124de4b98c6b1b1b78c3be` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:46.141405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86bfac3-8d8c-4b7c-98eb-7abb111a61a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=863637528a124de4b98c6b1b1b78c3be)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:46.146518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86bfac3-8d8c-4b7c-98eb-7abb111a61a7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:46.205710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1069/2125] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:45 2022] POST /identity/v3/users => generated 291 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:46.206655 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:46.216475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e6e87-acb8-4791-b42d-acee567655db admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:46.227699 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e6e87-acb8-4791-b42d-acee567655db admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:46.239671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e97295-d77a-4772-9121-fd8cc55ce6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:46.239671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e97295-d77a-4772-9121-fd8cc55ce6bd admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:46.239671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e97295-d77a-4772-9121-fd8cc55ce6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:46.241486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e97295-d77a-4772-9121-fd8cc55ce6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:46.241486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e97295-d77a-4772-9121-fd8cc55ce6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:46.244691 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e97295-d77a-4772-9121-fd8cc55ce6bd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:46.267655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1070/2126] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:48:46.269842 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e97295-d77a-4772-9121-fd8cc55ce6bd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:46.287600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e97295-d77a-4772-9121-fd8cc55ce6bd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:46.292706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee1cff-1d94-42a3-8990-c79581af7617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:46.293389 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee1cff-1d94-42a3-8990-c79581af7617 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:46.293666 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee1cff-1d94-42a3-8990-c79581af7617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:46.293936 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee1cff-1d94-42a3-8990-c79581af7617 admin admin] PATH_INFO: `/v3/projects/684ed0b6a32f4673b8efc9704a0652cb/users/556eee5850894701b69f708ba896a0cb/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:46.314505 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee1cff-1d94-42a3-8990-c79581af7617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=684ed0b6a32f4673b8efc9704a0652cb, user_id=556eee5850894701b69f708ba896a0cb, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:46.317200 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee1cff-1d94-42a3-8990-c79581af7617 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:46.997808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1071/2127] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:46 2022] PUT /identity/v3/projects/684ed0b6a32f4673b8efc9704a0652cb/users/556eee5850894701b69f708ba896a0cb/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:47.002543 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee1cff-1d94-42a3-8990-c79581af7617 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:47.108370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee1cff-1d94-42a3-8990-c79581af7617 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:47.114924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173ae0b-5f2e-437d-ab8d-9a481d4ffbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:47.115875 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173ae0b-5f2e-437d-ab8d-9a481d4ffbec admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:47.116402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173ae0b-5f2e-437d-ab8d-9a481d4ffbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:47.116692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173ae0b-5f2e-437d-ab8d-9a481d4ffbec admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:47.117342 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173ae0b-5f2e-437d-ab8d-9a481d4ffbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:47.119264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173ae0b-5f2e-437d-ab8d-9a481d4ffbec admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:47.137748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1072/2128] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:48:47.138949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173ae0b-5f2e-437d-ab8d-9a481d4ffbec admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:47.139866 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b86bfac3-8d8c-4b7c-98eb-7abb111a61a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '863637528a124de4b98c6b1b1b78c3be'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:47.156597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173ae0b-5f2e-437d-ab8d-9a481d4ffbec admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:47.165283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b574856-1a9a-4909-9858-ad83a05f9d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:47.165961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b574856-1a9a-4909-9858-ad83a05f9d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:47.166299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b574856-1a9a-4909-9858-ad83a05f9d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:47.166753 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b574856-1a9a-4909-9858-ad83a05f9d9e admin admin] PATH_INFO: `/v3/projects/684ed0b6a32f4673b8efc9704a0652cb/users/556eee5850894701b69f708ba896a0cb/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:47.170198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b574856-1a9a-4909-9858-ad83a05f9d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=684ed0b6a32f4673b8efc9704a0652cb, user_id=556eee5850894701b69f708ba896a0cb, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:47.170394 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b86bfac3-8d8c-4b7c-98eb-7abb111a61a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '863637528a124de4b98c6b1b1b78c3be'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:47.175929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b574856-1a9a-4909-9858-ad83a05f9d9e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:47.193808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1073/2129] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:47 2022] PUT /identity/v3/projects/684ed0b6a32f4673b8efc9704a0652cb/users/556eee5850894701b69f708ba896a0cb/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:47.205523 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c2e38-ab2f-43ab-91e2-d4ce00c432c0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:47.206122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c2e38-ab2f-43ab-91e2-d4ce00c432c0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:47.206616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c2e38-ab2f-43ab-91e2-d4ce00c432c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:47.249378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4b3c2e38-ab2f-43ab-91e2-d4ce00c432c0 None None] MFA Rules not processed for user `556eee5850894701b69f708ba896a0cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:47.251614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4b3c2e38-ab2f-43ab-91e2-d4ce00c432c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:47.286692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b86bfac3-8d8c-4b7c-98eb-7abb111a61a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '863637528a124de4b98c6b1b1b78c3be'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:47.290448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b86bfac3-8d8c-4b7c-98eb-7abb111a61a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '863637528a124de4b98c6b1b1b78c3be'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:47.301161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b86bfac3-8d8c-4b7c-98eb-7abb111a61a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '863637528a124de4b98c6b1b1b78c3be'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:47.312434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1057/2130] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:46 2022] DELETE /identity/v3/users/863637528a124de4b98c6b1b1b78c3be => generated 0 bytes in 1194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:47.314637 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767ae32-2069-493a-abf0-a473c2610e0c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:47.315342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767ae32-2069-493a-abf0-a473c2610e0c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:47.315807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767ae32-2069-493a-abf0-a473c2610e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:47.341858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7767ae32-2069-493a-abf0-a473c2610e0c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:47.344945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7767ae32-2069-493a-abf0-a473c2610e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:47.367591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1074/2131] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:47.379306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c2e38-ab2f-43ab-91e2-d4ce00c432c0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:47.380848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4b3c2e38-ab2f-43ab-91e2-d4ce00c432c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:47.417872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:47.477790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1058/2132] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:47.479082 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767ae32-2069-493a-abf0-a473c2610e0c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:47.481413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7767ae32-2069-493a-abf0-a473c2610e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:47.502407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c2e38-ab2f-43ab-91e2-d4ce00c432c0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:47.674320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767ae32-2069-493a-abf0-a473c2610e0c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:47.674623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023c56b-4655-4b29-b222-bba0266d43d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:47.674623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023c56b-4655-4b29-b222-bba0266d43d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:47.674623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023c56b-4655-4b29-b222-bba0266d43d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:47.674623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023c56b-4655-4b29-b222-bba0266d43d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:47.679089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55077ea8-1a68-4474-8ec8-80f01ad952bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:47.679716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55077ea8-1a68-4474-8ec8-80f01ad952bc service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:47.680021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55077ea8-1a68-4474-8ec8-80f01ad952bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:47.680261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55077ea8-1a68-4474-8ec8-80f01ad952bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:47.719069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55077ea8-1a68-4474-8ec8-80f01ad952bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:47.726716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55077ea8-1a68-4474-8ec8-80f01ad952bc service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:47.805820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1059/2133] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:47.809119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6017b60-92ca-4eee-9da9-8edc9ade5e8d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:47.810933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6017b60-92ca-4eee-9da9-8edc9ade5e8d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:47.811410 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6017b60-92ca-4eee-9da9-8edc9ade5e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:47.850671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023c56b-4655-4b29-b222-bba0266d43d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:47.856139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023c56b-4655-4b29-b222-bba0266d43d2 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:47.872296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f6017b60-92ca-4eee-9da9-8edc9ade5e8d None None] MFA Rules not processed for user `0b198ff6ed104068b38b993054617f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:47.873821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6017b60-92ca-4eee-9da9-8edc9ade5e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:47.883861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1075/2134] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:48:47.982045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:48.007799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1060/2135] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:48.095013 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023c56b-4655-4b29-b222-bba0266d43d2 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:48.108169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023c56b-4655-4b29-b222-bba0266d43d2 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:48.117785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e5a92-b3c2-43a1-b6f0-b743dd4be51c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:48.118438 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e5a92-b3c2-43a1-b6f0-b743dd4be51c service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:48.118707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e5a92-b3c2-43a1-b6f0-b743dd4be51c service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:48.118978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e5a92-b3c2-43a1-b6f0-b743dd4be51c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:48.124901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d27e5a92-b3c2-43a1-b6f0-b743dd4be51c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:48.227101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e5a92-b3c2-43a1-b6f0-b743dd4be51c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:48.228782 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e5a92-b3c2-43a1-b6f0-b743dd4be51c service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:48.244808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1076/2136] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:48.750893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6017b60-92ca-4eee-9da9-8edc9ade5e8d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:48.756658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6017b60-92ca-4eee-9da9-8edc9ade5e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:48.819342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6017b60-92ca-4eee-9da9-8edc9ade5e8d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:48.834741 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19776fd-af88-45ae-b2dd-200f85ebb583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:48.835538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19776fd-af88-45ae-b2dd-200f85ebb583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:48.835922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19776fd-af88-45ae-b2dd-200f85ebb583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:48.836299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19776fd-af88-45ae-b2dd-200f85ebb583 admin admin] PATH_INFO: `/v3/projects/4a20f4de05b84a1e84cfeaff7057be37` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:48.842543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19776fd-af88-45ae-b2dd-200f85ebb583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a20f4de05b84a1e84cfeaff7057be37)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:48.844786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19776fd-af88-45ae-b2dd-200f85ebb583 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:49.095231 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b19776fd-af88-45ae-b2dd-200f85ebb583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a20f4de05b84a1e84cfeaff7057be37', 'request_id': 'req-b19776fd-af88-45ae-b2dd-200f85ebb583'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:49.238469 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b19776fd-af88-45ae-b2dd-200f85ebb583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a20f4de05b84a1e84cfeaff7057be37', 'request_id': 'req-b19776fd-af88-45ae-b2dd-200f85ebb583'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:49.248190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b19776fd-af88-45ae-b2dd-200f85ebb583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a20f4de05b84a1e84cfeaff7057be37', 'request_id': 'req-b19776fd-af88-45ae-b2dd-200f85ebb583'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:49.265811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b19776fd-af88-45ae-b2dd-200f85ebb583 admin admin] The token cache is being invalidate because project 4a20f4de05b84a1e84cfeaff7057be37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:49.269466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b19776fd-af88-45ae-b2dd-200f85ebb583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:49.269466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b19776fd-af88-45ae-b2dd-200f85ebb583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:49.273066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1061/2137] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:48 2022] DELETE /identity/v3/projects/4a20f4de05b84a1e84cfeaff7057be37 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:49.277651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3245e8d-ccce-4578-a57b-8b59a9a15a1a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:49.278446 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3245e8d-ccce-4578-a57b-8b59a9a15a1a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:49.278588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3245e8d-ccce-4578-a57b-8b59a9a15a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:49.310033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c3245e8d-ccce-4578-a57b-8b59a9a15a1a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:49.313006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c3245e8d-ccce-4578-a57b-8b59a9a15a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:49.354280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:49.449883 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1077/2138] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:49.458575 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca45f4-b948-4f6d-9987-f46b82ba671a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:49.459848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca45f4-b948-4f6d-9987-f46b82ba671a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:49.460755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca45f4-b948-4f6d-9987-f46b82ba671a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:49.530238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-28ca45f4-b948-4f6d-9987-f46b82ba671a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:49.532807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-28ca45f4-b948-4f6d-9987-f46b82ba671a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:49.551472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1062/2139] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:49.557845 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3245e8d-ccce-4578-a57b-8b59a9a15a1a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:49.576187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3245e8d-ccce-4578-a57b-8b59a9a15a1a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:49.598450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ec190-daa5-46d6-90d2-b76c8811db98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:49.600035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ec190-daa5-46d6-90d2-b76c8811db98 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:49.609419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ec190-daa5-46d6-90d2-b76c8811db98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:49.609776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ec190-daa5-46d6-90d2-b76c8811db98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:49.610778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ec190-daa5-46d6-90d2-b76c8811db98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:48:49.617876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ec190-daa5-46d6-90d2-b76c8811db98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:49.619664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ec190-daa5-46d6-90d2-b76c8811db98 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:49.645733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1078/2140] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:48:49.649262 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca45f4-b948-4f6d-9987-f46b82ba671a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:49.671644 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca45f4-b948-4f6d-9987-f46b82ba671a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:49.680138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae82d98-cfd6-4e20-9160-b8677677ec0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:49.683499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae82d98-cfd6-4e20-9160-b8677677ec0b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:49.683499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae82d98-cfd6-4e20-9160-b8677677ec0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:49.683499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae82d98-cfd6-4e20-9160-b8677677ec0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:49.683499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae82d98-cfd6-4e20-9160-b8677677ec0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:49.686343 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae82d98-cfd6-4e20-9160-b8677677ec0b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:49.839381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1063/2141] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:49 2022] POST /identity/v3/projects => generated 375 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:48:49.840077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ec190-daa5-46d6-90d2-b76c8811db98 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:49.852230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ec190-daa5-46d6-90d2-b76c8811db98 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:50.023498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e7958-5585-4766-9440-b66ba9b56f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:50.024158 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e7958-5585-4766-9440-b66ba9b56f30 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:50.024560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e7958-5585-4766-9440-b66ba9b56f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:50.024867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e7958-5585-4766-9440-b66ba9b56f30 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:50.025756 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e7958-5585-4766-9440-b66ba9b56f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:50.028390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e7958-5585-4766-9440-b66ba9b56f30 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:50.188496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1079/2142] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:49 2022] POST /identity/v3/users => generated 288 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:48:50.190047 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae82d98-cfd6-4e20-9160-b8677677ec0b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:50.203761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae82d98-cfd6-4e20-9160-b8677677ec0b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:50.209865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a732dbf-c726-4133-88bf-15f08ad06bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:50.218376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a732dbf-c726-4133-88bf-15f08ad06bcd admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:50.219261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a732dbf-c726-4133-88bf-15f08ad06bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:50.219791 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a732dbf-c726-4133-88bf-15f08ad06bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:50.220643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a732dbf-c726-4133-88bf-15f08ad06bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:50.223498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a732dbf-c726-4133-88bf-15f08ad06bcd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:50.243753 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e7958-5585-4766-9440-b66ba9b56f30 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:50.265239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1064/2143] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:50.269308 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e7958-5585-4766-9440-b66ba9b56f30 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:50.285506 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca8cdf-3977-4141-a617-c09e997a494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:50.286360 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca8cdf-3977-4141-a617-c09e997a494c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:50.286519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca8cdf-3977-4141-a617-c09e997a494c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:50.286833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca8cdf-3977-4141-a617-c09e997a494c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:50.287519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca8cdf-3977-4141-a617-c09e997a494c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:50.291405 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca8cdf-3977-4141-a617-c09e997a494c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:50.317753 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1080/2144] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:50.319913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a732dbf-c726-4133-88bf-15f08ad06bcd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:50.334827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a732dbf-c726-4133-88bf-15f08ad06bcd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:50.340973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f228f18-0325-43bd-884d-fbddccb92fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:50.342078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f228f18-0325-43bd-884d-fbddccb92fae admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:50.342419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f228f18-0325-43bd-884d-fbddccb92fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:50.342696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f228f18-0325-43bd-884d-fbddccb92fae admin admin] PATH_INFO: `/v3/projects/bbf3c1c99fad4deab7ec4afd19968704/users/965f168d814249148103a80fa7f6664d/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:50.366442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f228f18-0325-43bd-884d-fbddccb92fae admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf3c1c99fad4deab7ec4afd19968704, user_id=965f168d814249148103a80fa7f6664d, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:50.369843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f228f18-0325-43bd-884d-fbddccb92fae admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:50.501793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1065/2145] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:50 2022] PUT /identity/v3/projects/bbf3c1c99fad4deab7ec4afd19968704/users/965f168d814249148103a80fa7f6664d/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:50.518185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1db1d4-8129-4c2e-aa02-d30f63a5be14 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:50.518185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1db1d4-8129-4c2e-aa02-d30f63a5be14 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:50.518432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1db1d4-8129-4c2e-aa02-d30f63a5be14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:50.585622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4e1db1d4-8129-4c2e-aa02-d30f63a5be14 None None] MFA Rules not processed for user `965f168d814249148103a80fa7f6664d`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:50.587070 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4e1db1d4-8129-4c2e-aa02-d30f63a5be14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:50.745552 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1081/2146] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:50.764194 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f228f18-0325-43bd-884d-fbddccb92fae admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:50.766567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4f228f18-0325-43bd-884d-fbddccb92fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:50.947279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f228f18-0325-43bd-884d-fbddccb92fae admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:50.954682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f66140-6192-4c44-8930-c89083bb7c66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:50.955315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f66140-6192-4c44-8930-c89083bb7c66 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:50.955659 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f66140-6192-4c44-8930-c89083bb7c66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:50.955929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f66140-6192-4c44-8930-c89083bb7c66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:50.968222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f66140-6192-4c44-8930-c89083bb7c66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:50.970434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f66140-6192-4c44-8930-c89083bb7c66 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:50.986325 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1066/2147] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:53.299257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1db1d4-8129-4c2e-aa02-d30f63a5be14 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:53.301216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4e1db1d4-8129-4c2e-aa02-d30f63a5be14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:53.341820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:53.402258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1db1d4-8129-4c2e-aa02-d30f63a5be14 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:53.409030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17753c9f-6366-4807-9909-649577ec4fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:53.409859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17753c9f-6366-4807-9909-649577ec4fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:53.410136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17753c9f-6366-4807-9909-649577ec4fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:53.410546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17753c9f-6366-4807-9909-649577ec4fbc admin admin] PATH_INFO: `/v3/users/59212cc1419848f189d8c582bce27ac2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:53.413006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17753c9f-6366-4807-9909-649577ec4fbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=59212cc1419848f189d8c582bce27ac2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:53.424905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17753c9f-6366-4807-9909-649577ec4fbc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:53.764516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17753c9f-6366-4807-9909-649577ec4fbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59212cc1419848f189d8c582bce27ac2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:53.765757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17753c9f-6366-4807-9909-649577ec4fbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59212cc1419848f189d8c582bce27ac2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:54.016317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17753c9f-6366-4807-9909-649577ec4fbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59212cc1419848f189d8c582bce27ac2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:54.034206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17753c9f-6366-4807-9909-649577ec4fbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59212cc1419848f189d8c582bce27ac2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:54.036935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc180e83-493d-4024-ad26-fca3eb4c0b2c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:54.036935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc180e83-493d-4024-ad26-fca3eb4c0b2c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:54.037550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc180e83-493d-4024-ad26-fca3eb4c0b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:54.064510 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17753c9f-6366-4807-9909-649577ec4fbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59212cc1419848f189d8c582bce27ac2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:54.074773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:54.079841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1082/2148] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:53 2022] DELETE /identity/v3/users/59212cc1419848f189d8c582bce27ac2 => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:54.086039 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da38c5-c18c-4929-b90e-7a7b790f4169 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:54.089614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da38c5-c18c-4929-b90e-7a7b790f4169 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:54.090019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da38c5-c18c-4929-b90e-7a7b790f4169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:54.097334 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cc180e83-493d-4024-ad26-fca3eb4c0b2c None None] MFA Rules not processed for user `a0dde6aa81204252b66b6e640b343ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:54.097334 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cc180e83-493d-4024-ad26-fca3eb4c0b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:54.122497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-82da38c5-c18c-4929-b90e-7a7b790f4169 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:54.123841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-82da38c5-c18c-4929-b90e-7a7b790f4169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:54.190059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1067/2149] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:54.249823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1083/2150] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:48:54.307893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc180e83-493d-4024-ad26-fca3eb4c0b2c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:54.310284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cc180e83-493d-4024-ad26-fca3eb4c0b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:54.565119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc180e83-493d-4024-ad26-fca3eb4c0b2c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:54.569764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b68d15-97bd-45ba-8e17-5c00ce82efbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:54.570652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b68d15-97bd-45ba-8e17-5c00ce82efbf service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:54.571006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b68d15-97bd-45ba-8e17-5c00ce82efbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:54.571351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b68d15-97bd-45ba-8e17-5c00ce82efbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:54.586854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b68d15-97bd-45ba-8e17-5c00ce82efbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:54.588598 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b68d15-97bd-45ba-8e17-5c00ce82efbf service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:54.603799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1068/2151] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:55.366038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da38c5-c18c-4929-b90e-7a7b790f4169 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:55.368534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-82da38c5-c18c-4929-b90e-7a7b790f4169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:55.419622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da38c5-c18c-4929-b90e-7a7b790f4169 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:55.432169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:55.433186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:55.433635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:55.434326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] PATH_INFO: `/v3/projects/68504220ca0d40af8157b20db9e4ba86` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:55.436115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68504220ca0d40af8157b20db9e4ba86)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:55.439923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:55.721363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:55.741230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68504220ca0d40af8157b20db9e4ba86', 'request_id': 'req-63413cb7-6bad-457e-9e83-317bcec8c004'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:55.926543 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68504220ca0d40af8157b20db9e4ba86', 'request_id': 'req-63413cb7-6bad-457e-9e83-317bcec8c004'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:55.935664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68504220ca0d40af8157b20db9e4ba86', 'request_id': 'req-63413cb7-6bad-457e-9e83-317bcec8c004'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:55.961516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] The token cache is being invalidate because project 68504220ca0d40af8157b20db9e4ba86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:55.962019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:55.963471 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:55.970802 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1084/2152] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:55 2022] DELETE /identity/v3/projects/68504220ca0d40af8157b20db9e4ba86 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:55.971821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b68d15-97bd-45ba-8e17-5c00ce82efbf service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:55.985960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b68d15-97bd-45ba-8e17-5c00ce82efbf service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:56.069351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:56.070022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:56.070345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:56.070606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] PATH_INFO: `/v3/users/39e88e65c9bf4ab6a0d6ce4187beb0b5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:56.071845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e88e65c9bf4ab6a0d6ce4187beb0b5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:56.073998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:56.459844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e88e65c9bf4ab6a0d6ce4187beb0b5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:56.467876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:56.473184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e88e65c9bf4ab6a0d6ce4187beb0b5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:56.643812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e88e65c9bf4ab6a0d6ce4187beb0b5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:56.647837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e88e65c9bf4ab6a0d6ce4187beb0b5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:56.666542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e88e65c9bf4ab6a0d6ce4187beb0b5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:56.687063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1069/2153] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:55 2022] DELETE /identity/v3/users/39e88e65c9bf4ab6a0d6ce4187beb0b5 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:57.178219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:57.179975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:57.195439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63413cb7-6bad-457e-9e83-317bcec8c004 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:57.284889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:57.286819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:57.287207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:57.287577 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] PATH_INFO: `/v3/users/ca4570b209464399b08c257d21ed1a55` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:57.291271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca4570b209464399b08c257d21ed1a55)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:57.293922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:57.381473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:57.383396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:57.408711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c514b-c7dc-4127-9b9a-2a6d01c0bc69 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:57.448635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c45d8e-70fa-4539-a9c6-2b4d947aa41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:57.451909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c45d8e-70fa-4539-a9c6-2b4d947aa41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:57.455152 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c45d8e-70fa-4539-a9c6-2b4d947aa41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:57.462475 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c45d8e-70fa-4539-a9c6-2b4d947aa41a admin admin] PATH_INFO: `/v3/users/965f168d814249148103a80fa7f6664d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:57.467851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c45d8e-70fa-4539-a9c6-2b4d947aa41a admin admin] RBAC: Authorizing `identity:delete_user(user_id=965f168d814249148103a80fa7f6664d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:57.477732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c45d8e-70fa-4539-a9c6-2b4d947aa41a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:57.559214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca4570b209464399b08c257d21ed1a55'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:57.560909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca4570b209464399b08c257d21ed1a55'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:57.771715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca4570b209464399b08c257d21ed1a55'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:57.781720 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca4570b209464399b08c257d21ed1a55'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:57.797791 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca4570b209464399b08c257d21ed1a55'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:57.804934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1085/2154] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:57 2022] DELETE /identity/v3/users/ca4570b209464399b08c257d21ed1a55 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:57.805882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:57.807505 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:57.882235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5c45d8e-70fa-4539-a9c6-2b4d947aa41a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '965f168d814249148103a80fa7f6664d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:57.895117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1f268-8120-4c75-936f-6e09cad514f1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:57.897434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5c45d8e-70fa-4539-a9c6-2b4d947aa41a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '965f168d814249148103a80fa7f6664d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:57.900704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:57.901430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:57.901709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:57.901985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] PATH_INFO: `/v3/projects/516977cec4c942bdb5f8bad3037b0876` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:57.903171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=516977cec4c942bdb5f8bad3037b0876)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:57.905209 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:58.018229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5c45d8e-70fa-4539-a9c6-2b4d947aa41a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '965f168d814249148103a80fa7f6664d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:58.018917 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5c45d8e-70fa-4539-a9c6-2b4d947aa41a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '965f168d814249148103a80fa7f6664d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:58.034370 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e5c45d8e-70fa-4539-a9c6-2b4d947aa41a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '965f168d814249148103a80fa7f6664d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:58.045904 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1070/2155] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:48:57 2022] DELETE /identity/v3/users/965f168d814249148103a80fa7f6664d => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:58.048243 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ffd9b-536b-4673-be05-13ccf7e0ec1f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:58.048739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ffd9b-536b-4673-be05-13ccf7e0ec1f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:58.049320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ffd9b-536b-4673-be05-13ccf7e0ec1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:58.072338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e53ffd9b-536b-4673-be05-13ccf7e0ec1f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:58.073924 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e53ffd9b-536b-4673-be05-13ccf7e0ec1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:58.121542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '516977cec4c942bdb5f8bad3037b0876', 'request_id': 'req-27561cb1-7145-4d4d-8d14-14845f9dd1e9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:58.144477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1071/2156] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:58.146722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894c288-75eb-4954-8191-e973c444499a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:58.147114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894c288-75eb-4954-8191-e973c444499a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:58.147665 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894c288-75eb-4954-8191-e973c444499a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:58.168295 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7894c288-75eb-4954-8191-e973c444499a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:58.169537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7894c288-75eb-4954-8191-e973c444499a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:58.250611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '516977cec4c942bdb5f8bad3037b0876', 'request_id': 'req-27561cb1-7145-4d4d-8d14-14845f9dd1e9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:58.251163 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1072/2157] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:58.251729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894c288-75eb-4954-8191-e973c444499a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:58.253288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7894c288-75eb-4954-8191-e973c444499a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:58.264002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '516977cec4c942bdb5f8bad3037b0876', 'request_id': 'req-27561cb1-7145-4d4d-8d14-14845f9dd1e9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:58.288900 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] The token cache is being invalidate because project 516977cec4c942bdb5f8bad3037b0876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:58.289127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:58.290542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:58.291324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1086/2158] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:57 2022] DELETE /identity/v3/projects/516977cec4c942bdb5f8bad3037b0876 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:58.295854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:58.297534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:58.365896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894c288-75eb-4954-8191-e973c444499a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:58.376878 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d78e3-46ff-4865-b8eb-ac53e2c93ecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:58.382680 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d78e3-46ff-4865-b8eb-ac53e2c93ecc service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:58.383731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d78e3-46ff-4865-b8eb-ac53e2c93ecc service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:58.384697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d78e3-46ff-4865-b8eb-ac53e2c93ecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:58.385496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27561cb1-7145-4d4d-8d14-14845f9dd1e9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:58.391259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c77d78e3-46ff-4865-b8eb-ac53e2c93ecc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:58.401849 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a355db7-53b1-4aed-9203-c2b3494203a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:58.402889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a355db7-53b1-4aed-9203-c2b3494203a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:58.403549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a355db7-53b1-4aed-9203-c2b3494203a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:58.403998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a355db7-53b1-4aed-9203-c2b3494203a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:58.406174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2a355db7-53b1-4aed-9203-c2b3494203a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:58.420826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a355db7-53b1-4aed-9203-c2b3494203a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:58.422587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a355db7-53b1-4aed-9203-c2b3494203a2 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:58.423020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d78e3-46ff-4865-b8eb-ac53e2c93ecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:58.426169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d78e3-46ff-4865-b8eb-ac53e2c93ecc service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:58.435161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1087/2159] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:58.436498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94816582-12d0-47f1-a301-169225582b97 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:58.436784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94816582-12d0-47f1-a301-169225582b97 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:58.437748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94816582-12d0-47f1-a301-169225582b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:58.449176 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1073/2160] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:48:58.461646 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-94816582-12d0-47f1-a301-169225582b97 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:58.462981 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-94816582-12d0-47f1-a301-169225582b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:58.474592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1088/2161] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:58.483123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89553b1-cee5-43a5-9fe2-16a302af8210 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:58.483803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89553b1-cee5-43a5-9fe2-16a302af8210 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:58.484317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89553b1-cee5-43a5-9fe2-16a302af8210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:58.515105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b89553b1-cee5-43a5-9fe2-16a302af8210 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:58.516932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b89553b1-cee5-43a5-9fe2-16a302af8210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:58.532249 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1074/2162] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:58.556239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94816582-12d0-47f1-a301-169225582b97 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:58.571276 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94816582-12d0-47f1-a301-169225582b97 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:58.581975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d910f-a26a-48d9-a363-b9889a1a38bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:58.583062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d910f-a26a-48d9-a363-b9889a1a38bf admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:58.583248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d910f-a26a-48d9-a363-b9889a1a38bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:58.585821 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d910f-a26a-48d9-a363-b9889a1a38bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:58.586918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d910f-a26a-48d9-a363-b9889a1a38bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:48:58.587437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d910f-a26a-48d9-a363-b9889a1a38bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:58.591047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d910f-a26a-48d9-a363-b9889a1a38bf admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:58.617248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1089/2163] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:48:58.622743 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89553b1-cee5-43a5-9fe2-16a302af8210 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:58.644064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89553b1-cee5-43a5-9fe2-16a302af8210 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:58.654839 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105f70f-b366-4907-913d-0f2158c82538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:58.656183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105f70f-b366-4907-913d-0f2158c82538 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:58.656601 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105f70f-b366-4907-913d-0f2158c82538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:58.657502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105f70f-b366-4907-913d-0f2158c82538 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:58.658783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105f70f-b366-4907-913d-0f2158c82538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:58.663319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105f70f-b366-4907-913d-0f2158c82538 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:58.787452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:58.799364 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1075/2164] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:48:58 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:48:58.806880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d910f-a26a-48d9-a363-b9889a1a38bf admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:58.844249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d910f-a26a-48d9-a363-b9889a1a38bf admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:58.915044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:48:58.958187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7562415-43c4-4700-a46c-858556317373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:58.960415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7562415-43c4-4700-a46c-858556317373 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:58.961353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7562415-43c4-4700-a46c-858556317373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:58.961353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7562415-43c4-4700-a46c-858556317373 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:58.961914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7562415-43c4-4700-a46c-858556317373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:58.965118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7562415-43c4-4700-a46c-858556317373 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:59.094179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1090/2165] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:48:58 2022] POST /identity/v3/users => generated 286 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:48:59.094543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105f70f-b366-4907-913d-0f2158c82538 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:59.106783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105f70f-b366-4907-913d-0f2158c82538 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:59.112610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bba7b-18e4-4339-85da-cc6cd5bf5f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:59.113259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bba7b-18e4-4339-85da-cc6cd5bf5f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:59.117375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bba7b-18e4-4339-85da-cc6cd5bf5f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:59.117375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bba7b-18e4-4339-85da-cc6cd5bf5f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:59.117375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bba7b-18e4-4339-85da-cc6cd5bf5f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:59.117375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bba7b-18e4-4339-85da-cc6cd5bf5f16 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:59.129298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1076/2166] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:48:59.133592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7562415-43c4-4700-a46c-858556317373 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:59.145305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7562415-43c4-4700-a46c-858556317373 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:59.151141 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58276d5a-0f8d-43c9-94a3-e19fce441601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:59.151770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58276d5a-0f8d-43c9-94a3-e19fce441601 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:59.152106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58276d5a-0f8d-43c9-94a3-e19fce441601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:59.152377 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58276d5a-0f8d-43c9-94a3-e19fce441601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:59.153033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58276d5a-0f8d-43c9-94a3-e19fce441601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:59.154851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58276d5a-0f8d-43c9-94a3-e19fce441601 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:59.170636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1091/2167] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:48:59.173687 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bba7b-18e4-4339-85da-cc6cd5bf5f16 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:59.177946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58276d5a-0f8d-43c9-94a3-e19fce441601 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:59.180100 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-58276d5a-0f8d-43c9-94a3-e19fce441601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:59.195670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58276d5a-0f8d-43c9-94a3-e19fce441601 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:59.203970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:59.203970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:59.203970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:59.203970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe admin admin] PATH_INFO: `/v3/projects/bbf3c1c99fad4deab7ec4afd19968704` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:59.205914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbf3c1c99fad4deab7ec4afd19968704)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:59.209530 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:59.229409 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bba7b-18e4-4339-85da-cc6cd5bf5f16 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:59.239155 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d00b7-fe5d-4fc0-9a4c-e69d91950970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:59.240341 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d00b7-fe5d-4fc0-9a4c-e69d91950970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:59.240823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d00b7-fe5d-4fc0-9a4c-e69d91950970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:59.241310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d00b7-fe5d-4fc0-9a4c-e69d91950970 admin admin] PATH_INFO: `/v3/projects/3a58131788a94091a90239807e46e789/users/f2fdef783c244114a4161e3a6489b672/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:59.273769 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d00b7-fe5d-4fc0-9a4c-e69d91950970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a58131788a94091a90239807e46e789, user_id=f2fdef783c244114a4161e3a6489b672, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:59.279700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d00b7-fe5d-4fc0-9a4c-e69d91950970 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:59.421715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1077/2168] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:48:59 2022] PUT /identity/v3/projects/3a58131788a94091a90239807e46e789/users/f2fdef783c244114a4161e3a6489b672/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:48:59.424973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d00b7-fe5d-4fc0-9a4c-e69d91950970 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:59.428195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e97d00b7-fe5d-4fc0-9a4c-e69d91950970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:59.435214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbf3c1c99fad4deab7ec4afd19968704', 'request_id': 'req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:59.543126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d00b7-fe5d-4fc0-9a4c-e69d91950970 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:48:59.624872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbf3c1c99fad4deab7ec4afd19968704', 'request_id': 'req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:59.637302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e8fc6-754a-486a-b915-d2320d75d09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:48:59.638101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e8fc6-754a-486a-b915-d2320d75d09a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:59.638481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e8fc6-754a-486a-b915-d2320d75d09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:59.638824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e8fc6-754a-486a-b915-d2320d75d09a admin admin] PATH_INFO: `/v3/projects/668ee817c28d4bda8e2bd04b4783973c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:59.640195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e8fc6-754a-486a-b915-d2320d75d09a admin admin] RBAC: Authorizing `identity:delete_project(project_id=668ee817c28d4bda8e2bd04b4783973c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:48:59.642419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e8fc6-754a-486a-b915-d2320d75d09a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:48:59.644782 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbf3c1c99fad4deab7ec4afd19968704', 'request_id': 'req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:59.678072 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe admin admin] The token cache is being invalidate because project bbf3c1c99fad4deab7ec4afd19968704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:48:59.682865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:59.682865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bf217c2d-a3d6-44e3-874c-b3e19ad7d4fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:59.697750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1092/2169] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:59 2022] DELETE /identity/v3/projects/bbf3c1c99fad4deab7ec4afd19968704 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:48:59.705715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c0887-8da6-4a88-b726-8dfcb02188dd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:59.710413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c0887-8da6-4a88-b726-8dfcb02188dd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:59.711181 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c0887-8da6-4a88-b726-8dfcb02188dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:59.773547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e32c0887-8da6-4a88-b726-8dfcb02188dd None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:59.775839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e32c0887-8da6-4a88-b726-8dfcb02188dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:59.811904 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1093/2170] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:59.814053 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a42e7-2020-41ee-a3f8-fe80d62279b9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:48:59.814382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a42e7-2020-41ee-a3f8-fe80d62279b9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:48:59.814547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a42e7-2020-41ee-a3f8-fe80d62279b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:48:59.847298 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7e9a42e7-2020-41ee-a3f8-fe80d62279b9 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:48:59.848514 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7e9a42e7-2020-41ee-a3f8-fe80d62279b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:48:59.876407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a30e8fc6-754a-486a-b915-d2320d75d09a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '668ee817c28d4bda8e2bd04b4783973c', 'request_id': 'req-a30e8fc6-754a-486a-b915-d2320d75d09a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:48:59.954844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1094/2171] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:48:59.955803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a42e7-2020-41ee-a3f8-fe80d62279b9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:48:59.963136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7e9a42e7-2020-41ee-a3f8-fe80d62279b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:00.006484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a30e8fc6-754a-486a-b915-d2320d75d09a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '668ee817c28d4bda8e2bd04b4783973c', 'request_id': 'req-a30e8fc6-754a-486a-b915-d2320d75d09a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:00.013455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a30e8fc6-754a-486a-b915-d2320d75d09a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '668ee817c28d4bda8e2bd04b4783973c', 'request_id': 'req-a30e8fc6-754a-486a-b915-d2320d75d09a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:00.027737 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a42e7-2020-41ee-a3f8-fe80d62279b9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:00.028227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a30e8fc6-754a-486a-b915-d2320d75d09a admin admin] The token cache is being invalidate because project 668ee817c28d4bda8e2bd04b4783973c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:49:00.028784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a30e8fc6-754a-486a-b915-d2320d75d09a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:00.029466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a30e8fc6-754a-486a-b915-d2320d75d09a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:00.031348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1078/2172] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:48:59 2022] DELETE /identity/v3/projects/668ee817c28d4bda8e2bd04b4783973c => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:00.032004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291b192-9a23-4906-ab38-351cbc13a014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:00.032347 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3f271-31fa-48d9-ae9a-a58019c0e0d5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:00.032764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291b192-9a23-4906-ab38-351cbc13a014 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:00.033102 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3f271-31fa-48d9-ae9a-a58019c0e0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:00.033199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291b192-9a23-4906-ab38-351cbc13a014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:00.033290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3f271-31fa-48d9-ae9a-a58019c0e0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:00.033365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291b192-9a23-4906-ab38-351cbc13a014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:00.035638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d291b192-9a23-4906-ab38-351cbc13a014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:00.049970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291b192-9a23-4906-ab38-351cbc13a014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:00.051495 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291b192-9a23-4906-ab38-351cbc13a014 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:00.055760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dca3f271-31fa-48d9-ae9a-a58019c0e0d5 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:00.056985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dca3f271-31fa-48d9-ae9a-a58019c0e0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:00.061983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1095/2173] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:48:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:49:00.062864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291b192-9a23-4906-ab38-351cbc13a014 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:00.064225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d291b192-9a23-4906-ab38-351cbc13a014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:00.069647 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1079/2174] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:00.071341 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3f271-31fa-48d9-ae9a-a58019c0e0d5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:00.079399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291b192-9a23-4906-ab38-351cbc13a014 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:00.080904 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3f271-31fa-48d9-ae9a-a58019c0e0d5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:00.085143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84b57e-0795-44fb-b432-c3eb66b866fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:00.085860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84b57e-0795-44fb-b432-c3eb66b866fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:00.086518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84b57e-0795-44fb-b432-c3eb66b866fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:00.086518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84b57e-0795-44fb-b432-c3eb66b866fd admin admin] PATH_INFO: `/v3/users/556eee5850894701b69f708ba896a0cb` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:00.087965 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84b57e-0795-44fb-b432-c3eb66b866fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=556eee5850894701b69f708ba896a0cb)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:00.089830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10c25b-df58-4356-9796-02afbae6902f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:00.090099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84b57e-0795-44fb-b432-c3eb66b866fd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:00.090515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10c25b-df58-4356-9796-02afbae6902f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:00.090781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10c25b-df58-4356-9796-02afbae6902f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:00.091124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10c25b-df58-4356-9796-02afbae6902f admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:00.092264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10c25b-df58-4356-9796-02afbae6902f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:49:00.093204 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10c25b-df58-4356-9796-02afbae6902f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:00.094781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10c25b-df58-4356-9796-02afbae6902f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:00.113823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1080/2175] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:49:00.114970 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10c25b-df58-4356-9796-02afbae6902f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:00.126444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10c25b-df58-4356-9796-02afbae6902f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:00.131866 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4f65d-5f39-493d-a960-189171f3c761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:00.132550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4f65d-5f39-493d-a960-189171f3c761 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:00.133195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4f65d-5f39-493d-a960-189171f3c761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:00.133479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4f65d-5f39-493d-a960-189171f3c761 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:00.134238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4f65d-5f39-493d-a960-189171f3c761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:00.137385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4f65d-5f39-493d-a960-189171f3c761 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:00.273618 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1081/2176] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:49:00 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:00.280298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4f65d-5f39-493d-a960-189171f3c761 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:00.395079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4f65d-5f39-493d-a960-189171f3c761 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:00.403662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c769cf-f50e-4c56-8ba3-37e3f317713f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:00.404589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c769cf-f50e-4c56-8ba3-37e3f317713f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:00.405535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c769cf-f50e-4c56-8ba3-37e3f317713f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:00.406784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c769cf-f50e-4c56-8ba3-37e3f317713f admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:00.407651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c769cf-f50e-4c56-8ba3-37e3f317713f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:00.410427 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c769cf-f50e-4c56-8ba3-37e3f317713f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:00.439948 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3d84b57e-0795-44fb-b432-c3eb66b866fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '556eee5850894701b69f708ba896a0cb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:00.440991 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3d84b57e-0795-44fb-b432-c3eb66b866fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '556eee5850894701b69f708ba896a0cb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:00.549075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:00.555128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3d84b57e-0795-44fb-b432-c3eb66b866fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '556eee5850894701b69f708ba896a0cb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:00.557450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1082/2177] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:49:00 2022] POST /identity/v3/users => generated 287 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:00.560903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c769cf-f50e-4c56-8ba3-37e3f317713f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:00.563233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d7c769cf-f50e-4c56-8ba3-37e3f317713f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:00.582915 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c769cf-f50e-4c56-8ba3-37e3f317713f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:00.593854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3d84b57e-0795-44fb-b432-c3eb66b866fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '556eee5850894701b69f708ba896a0cb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:00.594260 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f925d-fd55-422a-8571-d970337e0886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:00.598866 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f925d-fd55-422a-8571-d970337e0886 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:00.598866 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f925d-fd55-422a-8571-d970337e0886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:00.598866 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f925d-fd55-422a-8571-d970337e0886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:00.598866 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f925d-fd55-422a-8571-d970337e0886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:00.602807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f925d-fd55-422a-8571-d970337e0886 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:00.623456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3d84b57e-0795-44fb-b432-c3eb66b866fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '556eee5850894701b69f708ba896a0cb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:00.632952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1083/2178] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:00.634589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f925d-fd55-422a-8571-d970337e0886 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:00.640403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1096/2179] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:49:00 2022] DELETE /identity/v3/users/556eee5850894701b69f708ba896a0cb => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:00.655273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f925d-fd55-422a-8571-d970337e0886 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:00.772081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:00.845611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be7856-f521-4f27-bf88-cb9b5470bf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:00.846430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be7856-f521-4f27-bf88-cb9b5470bf9b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:00.846820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be7856-f521-4f27-bf88-cb9b5470bf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:00.847201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be7856-f521-4f27-bf88-cb9b5470bf9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:00.848377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be7856-f521-4f27-bf88-cb9b5470bf9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:00.852438 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be7856-f521-4f27-bf88-cb9b5470bf9b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:00.879746 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1084/2180] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:00.884715 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84b57e-0795-44fb-b432-c3eb66b866fd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:00.913442 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84b57e-0795-44fb-b432-c3eb66b866fd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:00.930381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b2a71-f8cd-41bf-be37-0122d814e0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:00.931277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b2a71-f8cd-41bf-be37-0122d814e0ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:00.931655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b2a71-f8cd-41bf-be37-0122d814e0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:00.932055 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b2a71-f8cd-41bf-be37-0122d814e0ac admin admin] PATH_INFO: `/v3/projects/baaaaf5f58244280b0e265aa43bc571b/users/7c2338046a594f9eb4f2fef030f1b82b/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:00.952785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b2a71-f8cd-41bf-be37-0122d814e0ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=baaaaf5f58244280b0e265aa43bc571b, user_id=7c2338046a594f9eb4f2fef030f1b82b, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:00.955819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b2a71-f8cd-41bf-be37-0122d814e0ac admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:01.068825 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1097/2181] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:00 2022] PUT /identity/v3/projects/baaaaf5f58244280b0e265aa43bc571b/users/7c2338046a594f9eb4f2fef030f1b82b/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:01.088673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46965274-af4d-4e24-a763-cf61fa7bcda6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:01.089020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46965274-af4d-4e24-a763-cf61fa7bcda6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:01.089435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46965274-af4d-4e24-a763-cf61fa7bcda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:01.146888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-46965274-af4d-4e24-a763-cf61fa7bcda6 None None] MFA Rules not processed for user `7c2338046a594f9eb4f2fef030f1b82b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:01.148454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-46965274-af4d-4e24-a763-cf61fa7bcda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:01.345186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1085/2182] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:01.359130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b2a71-f8cd-41bf-be37-0122d814e0ac admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:01.361317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bd3b2a71-f8cd-41bf-be37-0122d814e0ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:01.532393 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b2a71-f8cd-41bf-be37-0122d814e0ac admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:01.537652 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8341f5e-04bc-43d1-a779-422f8058743c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:01.538444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8341f5e-04bc-43d1-a779-422f8058743c service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:01.538772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8341f5e-04bc-43d1-a779-422f8058743c service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:01.539106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8341f5e-04bc-43d1-a779-422f8058743c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:01.550906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8341f5e-04bc-43d1-a779-422f8058743c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:01.557667 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8341f5e-04bc-43d1-a779-422f8058743c service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:01.570228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1098/2183] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:49:01.695490 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46965274-af4d-4e24-a763-cf61fa7bcda6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:01.695490 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-46965274-af4d-4e24-a763-cf61fa7bcda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:01.838812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46965274-af4d-4e24-a763-cf61fa7bcda6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:01.845408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29e69a-63c6-4664-8874-202c83a5e8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:01.845880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29e69a-63c6-4664-8874-202c83a5e8a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:01.846392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29e69a-63c6-4664-8874-202c83a5e8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:01.846704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29e69a-63c6-4664-8874-202c83a5e8a1 admin admin] PATH_INFO: `/v3/projects/684ed0b6a32f4673b8efc9704a0652cb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:01.850344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29e69a-63c6-4664-8874-202c83a5e8a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=684ed0b6a32f4673b8efc9704a0652cb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:01.853255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29e69a-63c6-4664-8874-202c83a5e8a1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:01.985616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:02.019501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6c29e69a-63c6-4664-8874-202c83a5e8a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '684ed0b6a32f4673b8efc9704a0652cb', 'request_id': 'req-6c29e69a-63c6-4664-8874-202c83a5e8a1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:02.115910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6c29e69a-63c6-4664-8874-202c83a5e8a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '684ed0b6a32f4673b8efc9704a0652cb', 'request_id': 'req-6c29e69a-63c6-4664-8874-202c83a5e8a1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:02.125851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6c29e69a-63c6-4664-8874-202c83a5e8a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '684ed0b6a32f4673b8efc9704a0652cb', 'request_id': 'req-6c29e69a-63c6-4664-8874-202c83a5e8a1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:02.141223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6c29e69a-63c6-4664-8874-202c83a5e8a1 admin admin] The token cache is being invalidate because project 684ed0b6a32f4673b8efc9704a0652cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:49:02.141511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6c29e69a-63c6-4664-8874-202c83a5e8a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:02.142474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6c29e69a-63c6-4664-8874-202c83a5e8a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:02.144044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1086/2184] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:49:01 2022] DELETE /identity/v3/projects/684ed0b6a32f4673b8efc9704a0652cb => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:02.149472 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5c1e3-4b38-48a0-b946-781820db2f4a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:02.149677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5c1e3-4b38-48a0-b946-781820db2f4a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:02.149949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5c1e3-4b38-48a0-b946-781820db2f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:02.173111 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e8c5c1e3-4b38-48a0-b946-781820db2f4a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:02.174904 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e8c5c1e3-4b38-48a0-b946-781820db2f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:02.260611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1099/2185] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:02.268159 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f824e-77cd-4b6a-9ef3-9e8539178eed None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:02.268484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f824e-77cd-4b6a-9ef3-9e8539178eed None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:02.268765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f824e-77cd-4b6a-9ef3-9e8539178eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:02.289684 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cd4f824e-77cd-4b6a-9ef3-9e8539178eed None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:02.290975 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cd4f824e-77cd-4b6a-9ef3-9e8539178eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:02.299841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1087/2186] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:02.304976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5c1e3-4b38-48a0-b946-781820db2f4a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:02.315927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5c1e3-4b38-48a0-b946-781820db2f4a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:02.323609 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da11f0a-e902-4228-887b-14b0f9c1ae68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:02.326718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da11f0a-e902-4228-887b-14b0f9c1ae68 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:02.326718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da11f0a-e902-4228-887b-14b0f9c1ae68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:02.326718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da11f0a-e902-4228-887b-14b0f9c1ae68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:02.326718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da11f0a-e902-4228-887b-14b0f9c1ae68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:49:02.326718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da11f0a-e902-4228-887b-14b0f9c1ae68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:02.327365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da11f0a-e902-4228-887b-14b0f9c1ae68 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:02.340596 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1100/2187] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:49:02.345765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f824e-77cd-4b6a-9ef3-9e8539178eed None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:02.355997 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f824e-77cd-4b6a-9ef3-9e8539178eed None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:02.361446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef957d73-18d8-412d-960b-41985cfbe05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:02.362115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef957d73-18d8-412d-960b-41985cfbe05b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:02.362503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef957d73-18d8-412d-960b-41985cfbe05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:02.362769 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef957d73-18d8-412d-960b-41985cfbe05b admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:02.363421 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef957d73-18d8-412d-960b-41985cfbe05b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:02.366015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef957d73-18d8-412d-960b-41985cfbe05b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:02.511014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da11f0a-e902-4228-887b-14b0f9c1ae68 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:02.515983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1088/2188] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:49:02 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:02.531971 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da11f0a-e902-4228-887b-14b0f9c1ae68 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:02.647370 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaed872-dcf8-4301-94f4-55751cc7560c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:02.658338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaed872-dcf8-4301-94f4-55751cc7560c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:02.658338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaed872-dcf8-4301-94f4-55751cc7560c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:02.756821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-edaed872-dcf8-4301-94f4-55751cc7560c None None] MFA Rules not processed for user `f2fdef783c244114a4161e3a6489b672`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:02.759250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-edaed872-dcf8-4301-94f4-55751cc7560c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:02.776942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd24b35-2ec7-4da7-b665-49831340ba3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:02.777987 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd24b35-2ec7-4da7-b665-49831340ba3d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:02.778529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd24b35-2ec7-4da7-b665-49831340ba3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:02.779025 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd24b35-2ec7-4da7-b665-49831340ba3d admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:02.780205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd24b35-2ec7-4da7-b665-49831340ba3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:02.788712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd24b35-2ec7-4da7-b665-49831340ba3d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:02.977565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1089/2189] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:49:02.978908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910abf1-4f5b-479d-8ee9-38dca33a0757 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:02.979182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910abf1-4f5b-479d-8ee9-38dca33a0757 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:02.979450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910abf1-4f5b-479d-8ee9-38dca33a0757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:03.006743 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1101/2190] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:49:02 2022] POST /identity/v3/users => generated 293 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:03.009019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd24b35-2ec7-4da7-b665-49831340ba3d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:03.012356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7fd24b35-2ec7-4da7-b665-49831340ba3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:03.068215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7910abf1-4f5b-479d-8ee9-38dca33a0757 None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:03.069564 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7910abf1-4f5b-479d-8ee9-38dca33a0757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:03.077074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd24b35-2ec7-4da7-b665-49831340ba3d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:03.105913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cd669-9bbf-4009-bfa0-70e9cfe2a355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:03.107442 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cd669-9bbf-4009-bfa0-70e9cfe2a355 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:03.108221 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cd669-9bbf-4009-bfa0-70e9cfe2a355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:03.108706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cd669-9bbf-4009-bfa0-70e9cfe2a355 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:03.109909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cd669-9bbf-4009-bfa0-70e9cfe2a355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:03.114326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cd669-9bbf-4009-bfa0-70e9cfe2a355 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:03.186511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1090/2191] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:03.188238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910abf1-4f5b-479d-8ee9-38dca33a0757 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:03.204941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:03.219416 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910abf1-4f5b-479d-8ee9-38dca33a0757 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:03.229879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d4406-bc90-4a1a-99f1-c29a3b82f731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:03.231229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d4406-bc90-4a1a-99f1-c29a3b82f731 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:03.231646 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d4406-bc90-4a1a-99f1-c29a3b82f731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:03.232179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d4406-bc90-4a1a-99f1-c29a3b82f731 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:03.233695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d4406-bc90-4a1a-99f1-c29a3b82f731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:03.237287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d4406-bc90-4a1a-99f1-c29a3b82f731 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:03.270238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1091/2192] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:03.270506 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d4406-bc90-4a1a-99f1-c29a3b82f731 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:03.275950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6c3d4406-bc90-4a1a-99f1-c29a3b82f731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:03.285541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1102/2193] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:49:03 2022] POST /identity/v3/projects => generated 371 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:03.286936 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cd669-9bbf-4009-bfa0-70e9cfe2a355 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:03.409256 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d4406-bc90-4a1a-99f1-c29a3b82f731 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:03.425329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cd669-9bbf-4009-bfa0-70e9cfe2a355 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:03.434753 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac3f55-9477-41d4-9bce-ed23674bf7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:03.435376 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac3f55-9477-41d4-9bce-ed23674bf7ab admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:03.435655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac3f55-9477-41d4-9bce-ed23674bf7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:03.435923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac3f55-9477-41d4-9bce-ed23674bf7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:03.436575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac3f55-9477-41d4-9bce-ed23674bf7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:03.438255 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac3f55-9477-41d4-9bce-ed23674bf7ab admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:03.459716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1103/2194] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:03.460619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac3f55-9477-41d4-9bce-ed23674bf7ab admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:03.478529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac3f55-9477-41d4-9bce-ed23674bf7ab admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:03.483876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14fba1-5ca1-4a07-8e56-4313a2393b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:03.485728 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14fba1-5ca1-4a07-8e56-4313a2393b82 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:03.485728 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14fba1-5ca1-4a07-8e56-4313a2393b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:03.485728 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14fba1-5ca1-4a07-8e56-4313a2393b82 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:03.485728 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14fba1-5ca1-4a07-8e56-4313a2393b82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:03.488117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14fba1-5ca1-4a07-8e56-4313a2393b82 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:03.521840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4e21d-aaf7-432d-a714-7702829b5d1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:03.522499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4e21d-aaf7-432d-a714-7702829b5d1c service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:03.522769 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4e21d-aaf7-432d-a714-7702829b5d1c service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:03.523046 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4e21d-aaf7-432d-a714-7702829b5d1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:03.602904 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4e21d-aaf7-432d-a714-7702829b5d1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:03.604577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4e21d-aaf7-432d-a714-7702829b5d1c service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:03.615879 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1092/2195] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:49:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:49:03.616848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1104/2196] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:49:03 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:03.617065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4e21d-aaf7-432d-a714-7702829b5d1c service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:03.618936 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14fba1-5ca1-4a07-8e56-4313a2393b82 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:03.629101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4e21d-aaf7-432d-a714-7702829b5d1c service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:03.635463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef667d1-dfd5-4ae1-9a2e-2d135c9781d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:03.636300 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef667d1-dfd5-4ae1-9a2e-2d135c9781d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:03.636623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef667d1-dfd5-4ae1-9a2e-2d135c9781d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:03.636959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef667d1-dfd5-4ae1-9a2e-2d135c9781d8 admin admin] PATH_INFO: `/v3/projects/b21daa5caa904f53baca6e110091baa2/users/8199c59b81d245e8bceb6b69dd662aa7/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:03.638449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14fba1-5ca1-4a07-8e56-4313a2393b82 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:03.650189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ab5c-c0f8-46ee-ae81-00413709ee3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:03.651787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ab5c-c0f8-46ee-ae81-00413709ee3a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:03.651960 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ab5c-c0f8-46ee-ae81-00413709ee3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:03.652333 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ab5c-c0f8-46ee-ae81-00413709ee3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:03.655261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ab5c-c0f8-46ee-ae81-00413709ee3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:03.656562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ab5c-c0f8-46ee-ae81-00413709ee3a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:03.684408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef667d1-dfd5-4ae1-9a2e-2d135c9781d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21daa5caa904f53baca6e110091baa2, user_id=8199c59b81d245e8bceb6b69dd662aa7, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:03.687169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef667d1-dfd5-4ae1-9a2e-2d135c9781d8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:03.693682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1105/2197] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:03.701580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ab5c-c0f8-46ee-ae81-00413709ee3a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:03.721523 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ab5c-c0f8-46ee-ae81-00413709ee3a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:03.740487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba94757-a00f-46e7-9f39-e3cbbe401938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:03.740487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba94757-a00f-46e7-9f39-e3cbbe401938 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:03.740487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba94757-a00f-46e7-9f39-e3cbbe401938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:03.740487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba94757-a00f-46e7-9f39-e3cbbe401938 admin admin] PATH_INFO: `/v3/projects/7f64599e7ba94e4daa8e5389a167ae49/users/cb20b5ed2e1848559356fb9ebd79a623/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:03.752273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba94757-a00f-46e7-9f39-e3cbbe401938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f64599e7ba94e4daa8e5389a167ae49, user_id=cb20b5ed2e1848559356fb9ebd79a623, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:03.757696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba94757-a00f-46e7-9f39-e3cbbe401938 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:03.874119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1106/2198] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:03 2022] PUT /identity/v3/projects/7f64599e7ba94e4daa8e5389a167ae49/users/cb20b5ed2e1848559356fb9ebd79a623/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:03.877322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad2003-571c-4c1c-9265-9e95c985ebf0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:03.877322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad2003-571c-4c1c-9265-9e95c985ebf0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:03.877322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad2003-571c-4c1c-9265-9e95c985ebf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:03.885419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1093/2199] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:03 2022] PUT /identity/v3/projects/b21daa5caa904f53baca6e110091baa2/users/8199c59b81d245e8bceb6b69dd662aa7/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:03.887420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef667d1-dfd5-4ae1-9a2e-2d135c9781d8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:03.947968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7fad2003-571c-4c1c-9265-9e95c985ebf0 None None] MFA Rules not processed for user `7c2338046a594f9eb4f2fef030f1b82b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:03.951069 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7fad2003-571c-4c1c-9265-9e95c985ebf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:04.131063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1107/2200] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:04.131063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dd10a-3652-44ed-bdc1-a26b114840ff None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:04.131063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dd10a-3652-44ed-bdc1-a26b114840ff None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:04.131458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dd10a-3652-44ed-bdc1-a26b114840ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:04.174499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef667d1-dfd5-4ae1-9a2e-2d135c9781d8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:04.199201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523f2d-ad2a-46fa-8634-1ab2a6764062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:04.200015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523f2d-ad2a-46fa-8634-1ab2a6764062 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:04.201741 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523f2d-ad2a-46fa-8634-1ab2a6764062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:04.202243 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523f2d-ad2a-46fa-8634-1ab2a6764062 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:04.205932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33523f2d-ad2a-46fa-8634-1ab2a6764062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:04.212789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33523f2d-ad2a-46fa-8634-1ab2a6764062 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:04.247392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1094/2201] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:04.264855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523f2d-ad2a-46fa-8634-1ab2a6764062 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:04.266663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-eb0dd10a-3652-44ed-bdc1-a26b114840ff None None] MFA Rules not processed for user `8199c59b81d245e8bceb6b69dd662aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:04.267899 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-eb0dd10a-3652-44ed-bdc1-a26b114840ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:04.279745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523f2d-ad2a-46fa-8634-1ab2a6764062 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:04.282993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:04.292888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607863d-ff5e-4d59-8ed2-5851a18f5350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:04.297195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607863d-ff5e-4d59-8ed2-5851a18f5350 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:04.297936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607863d-ff5e-4d59-8ed2-5851a18f5350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:04.299764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607863d-ff5e-4d59-8ed2-5851a18f5350 admin admin] PATH_INFO: `/v3/projects/7f64599e7ba94e4daa8e5389a167ae49/users/cb20b5ed2e1848559356fb9ebd79a623/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:04.307815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607863d-ff5e-4d59-8ed2-5851a18f5350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f64599e7ba94e4daa8e5389a167ae49, user_id=cb20b5ed2e1848559356fb9ebd79a623, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:04.310877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607863d-ff5e-4d59-8ed2-5851a18f5350 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:04.341216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1095/2202] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:04 2022] PUT /identity/v3/projects/7f64599e7ba94e4daa8e5389a167ae49/users/cb20b5ed2e1848559356fb9ebd79a623/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:04.392336 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1108/2203] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:04.396451 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607863d-ff5e-4d59-8ed2-5851a18f5350 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:04.573669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607863d-ff5e-4d59-8ed2-5851a18f5350 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:04.583628 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ee0b6-e913-491c-a1bb-31b3fa8d1d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:04.584678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ee0b6-e913-491c-a1bb-31b3fa8d1d03 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:04.585385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ee0b6-e913-491c-a1bb-31b3fa8d1d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:04.585985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ee0b6-e913-491c-a1bb-31b3fa8d1d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:04.587648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ee0b6-e913-491c-a1bb-31b3fa8d1d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:04.593166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ee0b6-e913-491c-a1bb-31b3fa8d1d03 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:04.799253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1096/2204] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:49:04 2022] POST /identity/v3/projects => generated 385 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:04.803646 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dd10a-3652-44ed-bdc1-a26b114840ff None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:04.829178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dd10a-3652-44ed-bdc1-a26b114840ff None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:04.898406 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:04.965537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305408b-15b4-45bd-850b-52f98f9ad64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:04.966189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305408b-15b4-45bd-850b-52f98f9ad64f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:04.966474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305408b-15b4-45bd-850b-52f98f9ad64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:04.966745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305408b-15b4-45bd-850b-52f98f9ad64f admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:04.971808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305408b-15b4-45bd-850b-52f98f9ad64f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:04.971808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305408b-15b4-45bd-850b-52f98f9ad64f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:05.088176 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1109/2205] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:49:04 2022] POST /identity/v3/users => generated 299 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:49:05.088656 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ee0b6-e913-491c-a1bb-31b3fa8d1d03 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:05.116854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ee0b6-e913-491c-a1bb-31b3fa8d1d03 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:05.129493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425fb08-abb2-4269-92b2-40dc52f8712c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:05.131224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425fb08-abb2-4269-92b2-40dc52f8712c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:05.132072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425fb08-abb2-4269-92b2-40dc52f8712c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:05.133407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425fb08-abb2-4269-92b2-40dc52f8712c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:05.134999 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425fb08-abb2-4269-92b2-40dc52f8712c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:05.138697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425fb08-abb2-4269-92b2-40dc52f8712c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:05.163347 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1097/2206] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:05.167957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305408b-15b4-45bd-850b-52f98f9ad64f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:05.189550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305408b-15b4-45bd-850b-52f98f9ad64f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:05.200394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcbe48-3dec-4906-84e0-511f0b280d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:05.201702 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcbe48-3dec-4906-84e0-511f0b280d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:05.202262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcbe48-3dec-4906-84e0-511f0b280d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:05.202790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcbe48-3dec-4906-84e0-511f0b280d72 admin admin] PATH_INFO: `/v3/projects/cc13195c088e4debb6631df013a3ec63/users/b3ee0417c0034cc58db2701ca7a06eff/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:05.239156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcbe48-3dec-4906-84e0-511f0b280d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc13195c088e4debb6631df013a3ec63, user_id=b3ee0417c0034cc58db2701ca7a06eff, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:05.244671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcbe48-3dec-4906-84e0-511f0b280d72 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:05.365517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1110/2207] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:05 2022] PUT /identity/v3/projects/cc13195c088e4debb6631df013a3ec63/users/b3ee0417c0034cc58db2701ca7a06eff/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:05.370809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425fb08-abb2-4269-92b2-40dc52f8712c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:05.435186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425fb08-abb2-4269-92b2-40dc52f8712c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:05.456562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:05.623781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36973a06-01d4-4341-9f11-42ce274de6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:05.624430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36973a06-01d4-4341-9f11-42ce274de6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:05.626902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36973a06-01d4-4341-9f11-42ce274de6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:05.626902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36973a06-01d4-4341-9f11-42ce274de6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:05.628279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36973a06-01d4-4341-9f11-42ce274de6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:05.634641 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36973a06-01d4-4341-9f11-42ce274de6a1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:05.656829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1098/2208] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:05.658550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcbe48-3dec-4906-84e0-511f0b280d72 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:05.687312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcbe48-3dec-4906-84e0-511f0b280d72 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:05.710466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447571fa-04ad-459c-8ca0-ef790bdc8a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:05.711158 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447571fa-04ad-459c-8ca0-ef790bdc8a93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:05.711433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447571fa-04ad-459c-8ca0-ef790bdc8a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:05.711707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447571fa-04ad-459c-8ca0-ef790bdc8a93 admin admin] PATH_INFO: `/v3/projects/cc13195c088e4debb6631df013a3ec63/users/b3ee0417c0034cc58db2701ca7a06eff/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:05.715294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447571fa-04ad-459c-8ca0-ef790bdc8a93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc13195c088e4debb6631df013a3ec63, user_id=b3ee0417c0034cc58db2701ca7a06eff, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:05.718289 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447571fa-04ad-459c-8ca0-ef790bdc8a93 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:05.747616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1111/2209] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:05 2022] PUT /identity/v3/projects/cc13195c088e4debb6631df013a3ec63/users/b3ee0417c0034cc58db2701ca7a06eff/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:05.760057 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68cc1-1a83-428a-b9a0-30a5fa11126e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:05.760412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68cc1-1a83-428a-b9a0-30a5fa11126e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:05.761542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68cc1-1a83-428a-b9a0-30a5fa11126e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:05.853501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-89b68cc1-1a83-428a-b9a0-30a5fa11126e None None] MFA Rules not processed for user `b3ee0417c0034cc58db2701ca7a06eff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:05.854768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-89b68cc1-1a83-428a-b9a0-30a5fa11126e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:06.005624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1099/2210] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:06.015567 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447571fa-04ad-459c-8ca0-ef790bdc8a93 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:06.017801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-447571fa-04ad-459c-8ca0-ef790bdc8a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:06.196583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447571fa-04ad-459c-8ca0-ef790bdc8a93 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:06.205438 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59aec9-cbf2-46b3-b60d-e654638ef28c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:06.206189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59aec9-cbf2-46b3-b60d-e654638ef28c service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:06.206534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59aec9-cbf2-46b3-b60d-e654638ef28c service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:06.206865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59aec9-cbf2-46b3-b60d-e654638ef28c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:06.233701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59aec9-cbf2-46b3-b60d-e654638ef28c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:06.236278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59aec9-cbf2-46b3-b60d-e654638ef28c service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:06.251120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1112/2211] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:49:06.293499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b68cc1-1a83-428a-b9a0-30a5fa11126e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:06.431926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b68cc1-1a83-428a-b9a0-30a5fa11126e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:06.437201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a997d7b-7fd8-471c-9016-ea2623c80816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:06.437855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a997d7b-7fd8-471c-9016-ea2623c80816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:06.438135 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a997d7b-7fd8-471c-9016-ea2623c80816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:06.438445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a997d7b-7fd8-471c-9016-ea2623c80816 admin admin] PATH_INFO: `/v3/users/8199c59b81d245e8bceb6b69dd662aa7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:06.439833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a997d7b-7fd8-471c-9016-ea2623c80816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8199c59b81d245e8bceb6b69dd662aa7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:06.441951 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a997d7b-7fd8-471c-9016-ea2623c80816 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:06.827044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a997d7b-7fd8-471c-9016-ea2623c80816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8199c59b81d245e8bceb6b69dd662aa7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:06.843774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a997d7b-7fd8-471c-9016-ea2623c80816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8199c59b81d245e8bceb6b69dd662aa7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:07.006332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a997d7b-7fd8-471c-9016-ea2623c80816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8199c59b81d245e8bceb6b69dd662aa7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:07.006977 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a997d7b-7fd8-471c-9016-ea2623c80816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8199c59b81d245e8bceb6b69dd662aa7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:07.020608 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a997d7b-7fd8-471c-9016-ea2623c80816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8199c59b81d245e8bceb6b69dd662aa7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:07.054801 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1100/2212] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:49:06 2022] DELETE /identity/v3/users/8199c59b81d245e8bceb6b69dd662aa7 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:07.060369 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9633f-401f-4cbc-87b6-5305cd138e10 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:07.060906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9633f-401f-4cbc-87b6-5305cd138e10 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:07.060906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9633f-401f-4cbc-87b6-5305cd138e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:07.103612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4be9633f-401f-4cbc-87b6-5305cd138e10 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:07.108544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4be9633f-401f-4cbc-87b6-5305cd138e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:07.190421 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570cbf1-270b-4465-b366-ade26d6fdb19 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:07.193694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570cbf1-270b-4465-b366-ade26d6fdb19 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:07.193694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570cbf1-270b-4465-b366-ade26d6fdb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:07.216168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:07.267637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1113/2213] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:07.282295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9633f-401f-4cbc-87b6-5305cd138e10 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:07.283619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4be9633f-401f-4cbc-87b6-5305cd138e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:07.304166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0570cbf1-270b-4465-b366-ade26d6fdb19 None None] MFA Rules not processed for user `cb20b5ed2e1848559356fb9ebd79a623`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:07.305368 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0570cbf1-270b-4465-b366-ade26d6fdb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:07.437020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9633f-401f-4cbc-87b6-5305cd138e10 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:07.442649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199af49a-e4db-4b02-8e88-7b0b9305ffd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:07.443378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199af49a-e4db-4b02-8e88-7b0b9305ffd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:07.443623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199af49a-e4db-4b02-8e88-7b0b9305ffd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:07.444076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199af49a-e4db-4b02-8e88-7b0b9305ffd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:07.459758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199af49a-e4db-4b02-8e88-7b0b9305ffd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:07.461766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199af49a-e4db-4b02-8e88-7b0b9305ffd0 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:07.479504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1101/2214] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:07.488206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1114/2215] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:49:07.494058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570cbf1-270b-4465-b366-ade26d6fdb19 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:07.495926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0570cbf1-270b-4465-b366-ade26d6fdb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:07.621620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:07.681954 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570cbf1-270b-4465-b366-ade26d6fdb19 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:07.701213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ecd2c-0ccc-4f63-8ac3-797eaf670303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:07.701213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ecd2c-0ccc-4f63-8ac3-797eaf670303 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:07.701213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ecd2c-0ccc-4f63-8ac3-797eaf670303 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:07.701213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ecd2c-0ccc-4f63-8ac3-797eaf670303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:07.701826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ae0ecd2c-0ccc-4f63-8ac3-797eaf670303 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:07.862677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ecd2c-0ccc-4f63-8ac3-797eaf670303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:07.864163 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ecd2c-0ccc-4f63-8ac3-797eaf670303 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:07.877608 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1102/2216] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:49:08.381834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199af49a-e4db-4b02-8e88-7b0b9305ffd0 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:08.383679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-199af49a-e4db-4b02-8e88-7b0b9305ffd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:08.414839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199af49a-e4db-4b02-8e88-7b0b9305ffd0 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:08.422686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:08.432681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:08.433089 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:08.433561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] PATH_INFO: `/v3/projects/b21daa5caa904f53baca6e110091baa2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:08.440388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21daa5caa904f53baca6e110091baa2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:08.444764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:08.653176 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21daa5caa904f53baca6e110091baa2', 'request_id': 'req-11e3340c-4257-447b-a36e-ed0a1b5b5570'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:08.763817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21daa5caa904f53baca6e110091baa2', 'request_id': 'req-11e3340c-4257-447b-a36e-ed0a1b5b5570'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:08.783484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21daa5caa904f53baca6e110091baa2', 'request_id': 'req-11e3340c-4257-447b-a36e-ed0a1b5b5570'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:08.805853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] The token cache is being invalidate because project b21daa5caa904f53baca6e110091baa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:49:08.805853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:08.806541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:08.813026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1115/2217] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:49:08 2022] DELETE /identity/v3/projects/b21daa5caa904f53baca6e110091baa2 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:08.816315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ecd2c-0ccc-4f63-8ac3-797eaf670303 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:08.878994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ecd2c-0ccc-4f63-8ac3-797eaf670303 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:09.021818 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:09.022431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:09.022714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:09.023003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] PATH_INFO: `/v3/users/b3ee0417c0034cc58db2701ca7a06eff` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:09.029648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3ee0417c0034cc58db2701ca7a06eff)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:09.033167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:09.041346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:09.812136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3ee0417c0034cc58db2701ca7a06eff'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:09.830558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3ee0417c0034cc58db2701ca7a06eff'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:09.955348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3ee0417c0034cc58db2701ca7a06eff'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:09.956108 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3ee0417c0034cc58db2701ca7a06eff'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:09.967161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3ee0417c0034cc58db2701ca7a06eff'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:09.978218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1103/2218] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:49:08 2022] DELETE /identity/v3/users/b3ee0417c0034cc58db2701ca7a06eff => generated 0 bytes in 1164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:10.086195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:10.715805 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:10.719169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:10.736656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3340c-4257-447b-a36e-ed0a1b5b5570 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:10.842823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc756345-19db-4cd7-a3d5-e198690e72b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:10.844544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc756345-19db-4cd7-a3d5-e198690e72b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:10.845068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc756345-19db-4cd7-a3d5-e198690e72b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:10.845347 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc756345-19db-4cd7-a3d5-e198690e72b4 admin admin] PATH_INFO: `/v3/projects/cc13195c088e4debb6631df013a3ec63` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:10.847032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc756345-19db-4cd7-a3d5-e198690e72b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc13195c088e4debb6631df013a3ec63)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:10.850026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc756345-19db-4cd7-a3d5-e198690e72b4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:10.984863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dc756345-19db-4cd7-a3d5-e198690e72b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc13195c088e4debb6631df013a3ec63', 'request_id': 'req-dc756345-19db-4cd7-a3d5-e198690e72b4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:11.073088 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dc756345-19db-4cd7-a3d5-e198690e72b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc13195c088e4debb6631df013a3ec63', 'request_id': 'req-dc756345-19db-4cd7-a3d5-e198690e72b4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:11.090466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:11.090466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:11.097372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dc756345-19db-4cd7-a3d5-e198690e72b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc13195c088e4debb6631df013a3ec63', 'request_id': 'req-dc756345-19db-4cd7-a3d5-e198690e72b4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:11.142523 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dc756345-19db-4cd7-a3d5-e198690e72b4 admin admin] The token cache is being invalidate because project cc13195c088e4debb6631df013a3ec63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:49:11.143196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dc756345-19db-4cd7-a3d5-e198690e72b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:11.148560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dc756345-19db-4cd7-a3d5-e198690e72b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:11.158213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1116/2219] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:49:10 2022] DELETE /identity/v3/projects/cc13195c088e4debb6631df013a3ec63 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:11.160069 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f542ea-eac7-4316-904e-eebb1ade6aa0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:11.160534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f542ea-eac7-4316-904e-eebb1ade6aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:11.162072 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f542ea-eac7-4316-904e-eebb1ade6aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:11.208598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:11.218657 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-05f542ea-eac7-4316-904e-eebb1ade6aa0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:11.220066 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-05f542ea-eac7-4316-904e-eebb1ade6aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:11.243221 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603a22a-e4a6-4b17-8673-7de0819e44a7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:11.322195 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1117/2220] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:49:11.329134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4eb2f-022c-4ba2-bc7e-ff3cff28123f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:11.330279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4eb2f-022c-4ba2-bc7e-ff3cff28123f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:11.331291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4eb2f-022c-4ba2-bc7e-ff3cff28123f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:11.334013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c81cff-e6c6-4ba0-82fd-eb75c928f631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:11.334813 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c81cff-e6c6-4ba0-82fd-eb75c928f631 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:11.335102 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c81cff-e6c6-4ba0-82fd-eb75c928f631 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:11.335377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c81cff-e6c6-4ba0-82fd-eb75c928f631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:11.337349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-77c81cff-e6c6-4ba0-82fd-eb75c928f631 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:11.365353 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:11.402446 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-85b4eb2f-022c-4ba2-bc7e-ff3cff28123f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:11.403869 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-85b4eb2f-022c-4ba2-bc7e-ff3cff28123f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:11.430602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1118/2221] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:49:11.435537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4eb2f-022c-4ba2-bc7e-ff3cff28123f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:11.449105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4eb2f-022c-4ba2-bc7e-ff3cff28123f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:11.462101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5083d-ff99-4a60-bdb1-e308ba410244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:11.462760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5083d-ff99-4a60-bdb1-e308ba410244 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:11.463035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5083d-ff99-4a60-bdb1-e308ba410244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:11.463304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5083d-ff99-4a60-bdb1-e308ba410244 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:11.463975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5083d-ff99-4a60-bdb1-e308ba410244 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:49:11.464273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5083d-ff99-4a60-bdb1-e308ba410244 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:11.469959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5083d-ff99-4a60-bdb1-e308ba410244 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:11.481128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c81cff-e6c6-4ba0-82fd-eb75c928f631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:11.483990 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c81cff-e6c6-4ba0-82fd-eb75c928f631 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:11.502106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1119/2223] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:49:11.508976 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1104/2223] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:49:11.509103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5083d-ff99-4a60-bdb1-e308ba410244 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:11.529251 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5083d-ff99-4a60-bdb1-e308ba410244 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:11.546309 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6391eaf-04fd-4f5e-aa21-06f3c24d4e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:11.546924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6391eaf-04fd-4f5e-aa21-06f3c24d4e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:11.547188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6391eaf-04fd-4f5e-aa21-06f3c24d4e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:11.550530 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6391eaf-04fd-4f5e-aa21-06f3c24d4e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:11.552708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6391eaf-04fd-4f5e-aa21-06f3c24d4e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:11.556820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6391eaf-04fd-4f5e-aa21-06f3c24d4e8d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:11.751875 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1120/2224] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:49:11 2022] POST /identity/v3/projects => generated 385 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:11.760784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c81cff-e6c6-4ba0-82fd-eb75c928f631 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:11.826952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c81cff-e6c6-4ba0-82fd-eb75c928f631 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:11.910093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:11.998328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b3340-459b-4185-884a-b099ff1b8656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:11.999103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b3340-459b-4185-884a-b099ff1b8656 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:11.999390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b3340-459b-4185-884a-b099ff1b8656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:11.999663 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b3340-459b-4185-884a-b099ff1b8656 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:12.000637 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b3340-459b-4185-884a-b099ff1b8656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:12.002825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b3340-459b-4185-884a-b099ff1b8656 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:12.177223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1105/2225] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:49:11 2022] POST /identity/v3/users => generated 293 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:12.177949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6391eaf-04fd-4f5e-aa21-06f3c24d4e8d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:12.199164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6391eaf-04fd-4f5e-aa21-06f3c24d4e8d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:12.206989 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430ea0-dccd-4de1-937c-49b224ce7800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:12.209067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430ea0-dccd-4de1-937c-49b224ce7800 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:12.209663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430ea0-dccd-4de1-937c-49b224ce7800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:12.211534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430ea0-dccd-4de1-937c-49b224ce7800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:12.224891 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2430ea0-dccd-4de1-937c-49b224ce7800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:12.234265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2430ea0-dccd-4de1-937c-49b224ce7800 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:12.266374 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1121/2226] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:12.271557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b3340-459b-4185-884a-b099ff1b8656 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:12.306717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b3340-459b-4185-884a-b099ff1b8656 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:12.341690 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8cf47-d513-458b-8f93-eeaf05ff2ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:12.351789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8cf47-d513-458b-8f93-eeaf05ff2ffc admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:12.351789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8cf47-d513-458b-8f93-eeaf05ff2ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:12.351789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8cf47-d513-458b-8f93-eeaf05ff2ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:12.351789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8cf47-d513-458b-8f93-eeaf05ff2ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:12.351789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8cf47-d513-458b-8f93-eeaf05ff2ffc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:12.377605 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1106/2227] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:12.383343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430ea0-dccd-4de1-937c-49b224ce7800 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:12.403838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430ea0-dccd-4de1-937c-49b224ce7800 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:12.427479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259f93b-2bda-4f5e-a618-b0ff4047cb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:12.432050 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259f93b-2bda-4f5e-a618-b0ff4047cb6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:12.432050 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259f93b-2bda-4f5e-a618-b0ff4047cb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:12.432050 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259f93b-2bda-4f5e-a618-b0ff4047cb6d admin admin] PATH_INFO: `/v3/projects/bda39c302ad048a39532ec6c3d1dc609/users/d83c7b3cf2d445038b59fc2515db85b1/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:12.517452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259f93b-2bda-4f5e-a618-b0ff4047cb6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda39c302ad048a39532ec6c3d1dc609, user_id=d83c7b3cf2d445038b59fc2515db85b1, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:12.523793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259f93b-2bda-4f5e-a618-b0ff4047cb6d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:12.734658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1122/2228] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:12 2022] PUT /identity/v3/projects/bda39c302ad048a39532ec6c3d1dc609/users/d83c7b3cf2d445038b59fc2515db85b1/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:12.746931 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3bfa6-de98-4285-8ad3-a221a93e82f2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:12.747828 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3bfa6-de98-4285-8ad3-a221a93e82f2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:12.747980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3bfa6-de98-4285-8ad3-a221a93e82f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:12.805751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6bb3bfa6-de98-4285-8ad3-a221a93e82f2 None None] MFA Rules not processed for user `d83c7b3cf2d445038b59fc2515db85b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:12.807719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6bb3bfa6-de98-4285-8ad3-a221a93e82f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:12.960458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1107/2229] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:12.965456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259f93b-2bda-4f5e-a618-b0ff4047cb6d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:12.978256 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:13.091214 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259f93b-2bda-4f5e-a618-b0ff4047cb6d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:13.113365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0720f-13ff-4c07-bc53-adfac39d16b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:13.115655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0720f-13ff-4c07-bc53-adfac39d16b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:13.116063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0720f-13ff-4c07-bc53-adfac39d16b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:13.116501 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0720f-13ff-4c07-bc53-adfac39d16b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:13.117333 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0720f-13ff-4c07-bc53-adfac39d16b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:13.120616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0720f-13ff-4c07-bc53-adfac39d16b7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:13.247761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1123/2230] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:49:12 2022] POST /identity/v3/projects => generated 385 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:13.251427 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3bfa6-de98-4285-8ad3-a221a93e82f2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:13.271748 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3bfa6-de98-4285-8ad3-a221a93e82f2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:13.366410 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:13.374903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79bb12-961b-4364-b88d-95abd0990a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:13.375573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79bb12-961b-4364-b88d-95abd0990a03 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:13.376042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79bb12-961b-4364-b88d-95abd0990a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:13.376697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79bb12-961b-4364-b88d-95abd0990a03 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:13.378212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79bb12-961b-4364-b88d-95abd0990a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:13.379811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79bb12-961b-4364-b88d-95abd0990a03 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:13.485452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1108/2231] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:49:13 2022] POST /identity/v3/users => generated 299 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:49:13.489632 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0720f-13ff-4c07-bc53-adfac39d16b7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:13.501764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0720f-13ff-4c07-bc53-adfac39d16b7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:13.507181 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffc15a-7fdb-4746-8604-69267b75ae7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:13.512924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffc15a-7fdb-4746-8604-69267b75ae7e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:13.512924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffc15a-7fdb-4746-8604-69267b75ae7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:13.512924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffc15a-7fdb-4746-8604-69267b75ae7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:13.512924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffc15a-7fdb-4746-8604-69267b75ae7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:13.512924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffc15a-7fdb-4746-8604-69267b75ae7e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:13.526810 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1124/2232] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:13.533482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79bb12-961b-4364-b88d-95abd0990a03 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:13.547642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79bb12-961b-4364-b88d-95abd0990a03 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:13.555209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26dd0b0-c1ce-482b-8f86-2d99847e05e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:13.556337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dd0b0-c1ce-482b-8f86-2d99847e05e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:13.556780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dd0b0-c1ce-482b-8f86-2d99847e05e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:13.561189 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dd0b0-c1ce-482b-8f86-2d99847e05e7 admin admin] PATH_INFO: `/v3/projects/20c6bbda897048259b9eaf49ea9b9a4c/users/09644be7ee3e4766bc21f579ae4cda93/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:13.585981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26dd0b0-c1ce-482b-8f86-2d99847e05e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c6bbda897048259b9eaf49ea9b9a4c, user_id=09644be7ee3e4766bc21f579ae4cda93, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:13.588996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26dd0b0-c1ce-482b-8f86-2d99847e05e7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:13.667142 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1109/2233] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:13 2022] PUT /identity/v3/projects/20c6bbda897048259b9eaf49ea9b9a4c/users/09644be7ee3e4766bc21f579ae4cda93/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:13.674596 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffc15a-7fdb-4746-8604-69267b75ae7e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:13.687623 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffc15a-7fdb-4746-8604-69267b75ae7e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:13.762982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:13.811638 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd64504-0440-409f-9e67-eccf007b4496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:13.812505 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd64504-0440-409f-9e67-eccf007b4496 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:13.812899 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd64504-0440-409f-9e67-eccf007b4496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:13.813293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd64504-0440-409f-9e67-eccf007b4496 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:13.815612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd64504-0440-409f-9e67-eccf007b4496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:13.815973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd64504-0440-409f-9e67-eccf007b4496 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:13.849885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1125/2234] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:13.852155 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26dd0b0-c1ce-482b-8f86-2d99847e05e7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:13.901602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26dd0b0-c1ce-482b-8f86-2d99847e05e7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:13.927415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce7626-51a4-4459-86d2-6a71103187a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:13.928858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce7626-51a4-4459-86d2-6a71103187a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:13.929601 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce7626-51a4-4459-86d2-6a71103187a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:13.930134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce7626-51a4-4459-86d2-6a71103187a6 admin admin] PATH_INFO: `/v3/projects/20c6bbda897048259b9eaf49ea9b9a4c/users/09644be7ee3e4766bc21f579ae4cda93/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:13.933412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce7626-51a4-4459-86d2-6a71103187a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c6bbda897048259b9eaf49ea9b9a4c, user_id=09644be7ee3e4766bc21f579ae4cda93, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:13.939711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce7626-51a4-4459-86d2-6a71103187a6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:13.962213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1110/2235] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:13 2022] PUT /identity/v3/projects/20c6bbda897048259b9eaf49ea9b9a4c/users/09644be7ee3e4766bc21f579ae4cda93/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:13.973218 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a81e74-c04c-442e-b67c-a08663959b8c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:13.974369 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a81e74-c04c-442e-b67c-a08663959b8c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:13.974466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a81e74-c04c-442e-b67c-a08663959b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:14.030768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b2a81e74-c04c-442e-b67c-a08663959b8c None None] MFA Rules not processed for user `09644be7ee3e4766bc21f579ae4cda93`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:14.032527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b2a81e74-c04c-442e-b67c-a08663959b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:14.072607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:14.186967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1126/2236] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:14.204978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce7626-51a4-4459-86d2-6a71103187a6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:14.209830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3fce7626-51a4-4459-86d2-6a71103187a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:14.357300 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce7626-51a4-4459-86d2-6a71103187a6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:14.380521 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4a65c-8387-4b4e-adcd-8baae8f30c31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:14.383100 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4a65c-8387-4b4e-adcd-8baae8f30c31 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:14.383523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4a65c-8387-4b4e-adcd-8baae8f30c31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:14.385134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4a65c-8387-4b4e-adcd-8baae8f30c31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:14.426667 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff4a65c-8387-4b4e-adcd-8baae8f30c31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:14.428152 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff4a65c-8387-4b4e-adcd-8baae8f30c31 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:14.457059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1111/2237] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:49:14.887844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a81e74-c04c-442e-b67c-a08663959b8c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:15.037460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a81e74-c04c-442e-b67c-a08663959b8c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:15.048895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:15.051731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:15.051731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:15.052102 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] PATH_INFO: `/v3/users/d83c7b3cf2d445038b59fc2515db85b1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:15.054916 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d83c7b3cf2d445038b59fc2515db85b1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:15.059625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:15.350644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd83c7b3cf2d445038b59fc2515db85b1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:15.351714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd83c7b3cf2d445038b59fc2515db85b1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:15.355582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:15.449515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd83c7b3cf2d445038b59fc2515db85b1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:15.458481 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd83c7b3cf2d445038b59fc2515db85b1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:15.479779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd83c7b3cf2d445038b59fc2515db85b1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:15.491028 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1127/2238] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:49:14 2022] DELETE /identity/v3/users/d83c7b3cf2d445038b59fc2515db85b1 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:15.497534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ba231-dca8-4d2b-8cb3-cec158eb16fb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:15.498198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ba231-dca8-4d2b-8cb3-cec158eb16fb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:15.498605 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ba231-dca8-4d2b-8cb3-cec158eb16fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:15.545297 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6e8ba231-dca8-4d2b-8cb3-cec158eb16fb None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:15.547237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6e8ba231-dca8-4d2b-8cb3-cec158eb16fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:15.669264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1112/2239] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:15.676530 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:15.678107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:15.755910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e8748-635f-4f07-968f-a424b4f34df6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:15.760990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ec135-f634-42d7-bb87-1788648a7172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:15.761982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ec135-f634-42d7-bb87-1788648a7172 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:15.762326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ec135-f634-42d7-bb87-1788648a7172 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:15.762608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ec135-f634-42d7-bb87-1788648a7172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:15.778769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ec135-f634-42d7-bb87-1788648a7172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:15.780341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ec135-f634-42d7-bb87-1788648a7172 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:15.795818 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1128/2240] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:49:16.328490 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ba231-dca8-4d2b-8cb3-cec158eb16fb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:16.331183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6e8ba231-dca8-4d2b-8cb3-cec158eb16fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:16.363652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ba231-dca8-4d2b-8cb3-cec158eb16fb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:16.387017 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:16.388166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:16.389166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:16.389783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] PATH_INFO: `/v3/projects/bda39c302ad048a39532ec6c3d1dc609` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:16.392269 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bda39c302ad048a39532ec6c3d1dc609)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:16.398200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:16.540436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bda39c302ad048a39532ec6c3d1dc609', 'request_id': 'req-81e944be-8a1d-4543-9599-d91cdee101f3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:16.545066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:16.629137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bda39c302ad048a39532ec6c3d1dc609', 'request_id': 'req-81e944be-8a1d-4543-9599-d91cdee101f3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:16.636859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bda39c302ad048a39532ec6c3d1dc609', 'request_id': 'req-81e944be-8a1d-4543-9599-d91cdee101f3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:16.663231 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] The token cache is being invalidate because project bda39c302ad048a39532ec6c3d1dc609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:49:16.663534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:16.664234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:16.668202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1113/2241] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:49:16 2022] DELETE /identity/v3/projects/bda39c302ad048a39532ec6c3d1dc609 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:16.672363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ec135-f634-42d7-bb87-1788648a7172 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:16.686431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ec135-f634-42d7-bb87-1788648a7172 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:16.839507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18d69b-56eb-4b10-91a5-361aae762087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:16.840198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18d69b-56eb-4b10-91a5-361aae762087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:16.840470 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18d69b-56eb-4b10-91a5-361aae762087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:16.840950 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18d69b-56eb-4b10-91a5-361aae762087 admin admin] PATH_INFO: `/v3/users/09644be7ee3e4766bc21f579ae4cda93` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:16.842504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18d69b-56eb-4b10-91a5-361aae762087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09644be7ee3e4766bc21f579ae4cda93)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:16.844800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18d69b-56eb-4b10-91a5-361aae762087 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:17.174026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ae18d69b-56eb-4b10-91a5-361aae762087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09644be7ee3e4766bc21f579ae4cda93'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:17.175107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ae18d69b-56eb-4b10-91a5-361aae762087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09644be7ee3e4766bc21f579ae4cda93'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:17.263441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ae18d69b-56eb-4b10-91a5-361aae762087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09644be7ee3e4766bc21f579ae4cda93'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:17.274833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ae18d69b-56eb-4b10-91a5-361aae762087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09644be7ee3e4766bc21f579ae4cda93'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:17.289867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:17.300273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ae18d69b-56eb-4b10-91a5-361aae762087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09644be7ee3e4766bc21f579ae4cda93'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:17.333587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1129/2242] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:49:16 2022] DELETE /identity/v3/users/09644be7ee3e4766bc21f579ae4cda93 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:18.248693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:18.251482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:18.300912 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e944be-8a1d-4543-9599-d91cdee101f3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:18.579914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f93a35-4041-4a56-81b9-4739f082f371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:18.583417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f93a35-4041-4a56-81b9-4739f082f371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:18.583694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f93a35-4041-4a56-81b9-4739f082f371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:18.583968 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f93a35-4041-4a56-81b9-4739f082f371 admin admin] PATH_INFO: `/v3/projects/20c6bbda897048259b9eaf49ea9b9a4c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:18.592149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f93a35-4041-4a56-81b9-4739f082f371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c6bbda897048259b9eaf49ea9b9a4c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:18.594179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f93a35-4041-4a56-81b9-4739f082f371 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:18.890016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3f93a35-4041-4a56-81b9-4739f082f371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c6bbda897048259b9eaf49ea9b9a4c', 'request_id': 'req-b3f93a35-4041-4a56-81b9-4739f082f371'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:18.985241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3f93a35-4041-4a56-81b9-4739f082f371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c6bbda897048259b9eaf49ea9b9a4c', 'request_id': 'req-b3f93a35-4041-4a56-81b9-4739f082f371'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:18.995032 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3f93a35-4041-4a56-81b9-4739f082f371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c6bbda897048259b9eaf49ea9b9a4c', 'request_id': 'req-b3f93a35-4041-4a56-81b9-4739f082f371'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:19.022960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3f93a35-4041-4a56-81b9-4739f082f371 admin admin] The token cache is being invalidate because project 20c6bbda897048259b9eaf49ea9b9a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:49:19.023588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3f93a35-4041-4a56-81b9-4739f082f371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:19.025426 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3f93a35-4041-4a56-81b9-4739f082f371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:19.036273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1114/2243] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:49:18 2022] DELETE /identity/v3/projects/20c6bbda897048259b9eaf49ea9b9a4c => generated 0 bytes in 783 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:19.042314 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133df13-111a-4c5e-807a-7053ef524b1a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:19.043284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133df13-111a-4c5e-807a-7053ef524b1a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:19.043782 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133df13-111a-4c5e-807a-7053ef524b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:19.101479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5133df13-111a-4c5e-807a-7053ef524b1a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:19.104215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5133df13-111a-4c5e-807a-7053ef524b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:19.251296 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1130/2244] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:19.261919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f55aac-1cdf-4c9e-8fa9-883d450515e1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:19.264039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f55aac-1cdf-4c9e-8fa9-883d450515e1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:19.264039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f55aac-1cdf-4c9e-8fa9-883d450515e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:19.347738 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b7f55aac-1cdf-4c9e-8fa9-883d450515e1 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:19.350190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b7f55aac-1cdf-4c9e-8fa9-883d450515e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:19.382401 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1115/2245] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:19.384840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133df13-111a-4c5e-807a-7053ef524b1a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:19.398516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133df13-111a-4c5e-807a-7053ef524b1a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:19.404047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78225222-ba33-4019-8fcc-4d1cbe2a1981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:19.404733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78225222-ba33-4019-8fcc-4d1cbe2a1981 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:19.405014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78225222-ba33-4019-8fcc-4d1cbe2a1981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:19.405285 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78225222-ba33-4019-8fcc-4d1cbe2a1981 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:19.405952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78225222-ba33-4019-8fcc-4d1cbe2a1981 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:49:19.409764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78225222-ba33-4019-8fcc-4d1cbe2a1981 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:19.409764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78225222-ba33-4019-8fcc-4d1cbe2a1981 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:19.427496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1131/2246] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:49:19.433129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f55aac-1cdf-4c9e-8fa9-883d450515e1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:19.448381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f55aac-1cdf-4c9e-8fa9-883d450515e1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:19.457133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a81c0-0157-468b-ad5a-888b718accc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:19.458832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a81c0-0157-468b-ad5a-888b718accc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:19.459717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a81c0-0157-468b-ad5a-888b718accc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:19.460512 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a81c0-0157-468b-ad5a-888b718accc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:19.461671 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a81c0-0157-468b-ad5a-888b718accc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:19.465407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a81c0-0157-468b-ad5a-888b718accc6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:19.644469 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1116/2247] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:49:19 2022] POST /identity/v3/projects => generated 387 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:19.649299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78225222-ba33-4019-8fcc-4d1cbe2a1981 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:19.664126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78225222-ba33-4019-8fcc-4d1cbe2a1981 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:19.779521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fa3c6-c554-47e5-aede-e98fbe7dde5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:19.780257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fa3c6-c554-47e5-aede-e98fbe7dde5a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:19.780602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fa3c6-c554-47e5-aede-e98fbe7dde5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:19.780958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fa3c6-c554-47e5-aede-e98fbe7dde5a admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:19.781670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fa3c6-c554-47e5-aede-e98fbe7dde5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:19.783804 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fa3c6-c554-47e5-aede-e98fbe7dde5a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:19.900477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1132/2248] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:49:19 2022] POST /identity/v3/users => generated 294 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:19.905658 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a81c0-0157-468b-ad5a-888b718accc6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:19.924296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a81c0-0157-468b-ad5a-888b718accc6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:19.933652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8865e8-ae22-4535-a59c-960f678683d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:19.935031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8865e8-ae22-4535-a59c-960f678683d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:19.935446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8865e8-ae22-4535-a59c-960f678683d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:19.936171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8865e8-ae22-4535-a59c-960f678683d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:19.937467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8865e8-ae22-4535-a59c-960f678683d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:19.940238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8865e8-ae22-4535-a59c-960f678683d2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:19.953877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1117/2249] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:19.960025 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fa3c6-c554-47e5-aede-e98fbe7dde5a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:19.971088 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fa3c6-c554-47e5-aede-e98fbe7dde5a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:19.979455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb1bf5-2f3a-49ff-bc53-481511412ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:19.980668 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb1bf5-2f3a-49ff-bc53-481511412ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:19.981141 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb1bf5-2f3a-49ff-bc53-481511412ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:19.981614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb1bf5-2f3a-49ff-bc53-481511412ce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:19.983147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb1bf5-2f3a-49ff-bc53-481511412ce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:19.985835 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb1bf5-2f3a-49ff-bc53-481511412ce4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:20.009791 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1133/2250] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:49:20.011270 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8865e8-ae22-4535-a59c-960f678683d2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:20.022180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8865e8-ae22-4535-a59c-960f678683d2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:20.027291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1814d06-4045-47b4-8dd1-7d73e8ba9ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:20.028000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1814d06-4045-47b4-8dd1-7d73e8ba9ead admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:20.028272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1814d06-4045-47b4-8dd1-7d73e8ba9ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:20.028545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1814d06-4045-47b4-8dd1-7d73e8ba9ead admin admin] PATH_INFO: `/v3/projects/f4391e7d8e4044e49433c2fbc1f1b94f/users/a518d39c799842c9bd30e38fc61b9cee/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:20.047593 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1814d06-4045-47b4-8dd1-7d73e8ba9ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4391e7d8e4044e49433c2fbc1f1b94f, user_id=a518d39c799842c9bd30e38fc61b9cee, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:20.050586 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1814d06-4045-47b4-8dd1-7d73e8ba9ead admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:20.134345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1118/2251] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:20 2022] PUT /identity/v3/projects/f4391e7d8e4044e49433c2fbc1f1b94f/users/a518d39c799842c9bd30e38fc61b9cee/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:20.153688 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d6068-b6c0-4837-9ae1-ab0478fbc7d0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:20.154564 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d6068-b6c0-4837-9ae1-ab0478fbc7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:20.154794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d6068-b6c0-4837-9ae1-ab0478fbc7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:20.211187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e04d6068-b6c0-4837-9ae1-ab0478fbc7d0 None None] MFA Rules not processed for user `a518d39c799842c9bd30e38fc61b9cee`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:20.213294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e04d6068-b6c0-4837-9ae1-ab0478fbc7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:20.296004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:20.340911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1134/2252] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:49:20.347225 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1814d06-4045-47b4-8dd1-7d73e8ba9ead admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:20.363487 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:20.433030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1814d06-4045-47b4-8dd1-7d73e8ba9ead admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:20.438790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69573baf-67b2-4f0e-8ecc-f65060de5dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:20.439424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69573baf-67b2-4f0e-8ecc-f65060de5dec admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:20.439702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69573baf-67b2-4f0e-8ecc-f65060de5dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:20.439973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69573baf-67b2-4f0e-8ecc-f65060de5dec admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:20.440636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69573baf-67b2-4f0e-8ecc-f65060de5dec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:20.443324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69573baf-67b2-4f0e-8ecc-f65060de5dec admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:20.631767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d6068-b6c0-4837-9ae1-ab0478fbc7d0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:20.632297 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1119/2253] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:49:20 2022] POST /identity/v3/projects => generated 387 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:20.652649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d6068-b6c0-4837-9ae1-ab0478fbc7d0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:20.788207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486a7f3-3abd-4387-b1e6-bb5659abcfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:20.791624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486a7f3-3abd-4387-b1e6-bb5659abcfe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:20.792563 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486a7f3-3abd-4387-b1e6-bb5659abcfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:20.793272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486a7f3-3abd-4387-b1e6-bb5659abcfe7 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:20.794547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486a7f3-3abd-4387-b1e6-bb5659abcfe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:20.798612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486a7f3-3abd-4387-b1e6-bb5659abcfe7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:20.924253 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1135/2254] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:49:20 2022] POST /identity/v3/users => generated 294 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:49:20.929490 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69573baf-67b2-4f0e-8ecc-f65060de5dec admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:20.952691 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69573baf-67b2-4f0e-8ecc-f65060de5dec admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:20.981634 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cc21c-30a4-46d6-a3d2-9b7c4bb32200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:20.983124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cc21c-30a4-46d6-a3d2-9b7c4bb32200 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:20.990447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cc21c-30a4-46d6-a3d2-9b7c4bb32200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:20.991123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cc21c-30a4-46d6-a3d2-9b7c4bb32200 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:20.992337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cc21c-30a4-46d6-a3d2-9b7c4bb32200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:20.994445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cc21c-30a4-46d6-a3d2-9b7c4bb32200 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:21.013072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1120/2255] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:21.014378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486a7f3-3abd-4387-b1e6-bb5659abcfe7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:21.029992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486a7f3-3abd-4387-b1e6-bb5659abcfe7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:21.036951 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bb30e-fb57-4db6-aca8-bbed78e40452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:21.037586 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bb30e-fb57-4db6-aca8-bbed78e40452 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:21.038908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bb30e-fb57-4db6-aca8-bbed78e40452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:21.038908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bb30e-fb57-4db6-aca8-bbed78e40452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:21.039269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bb30e-fb57-4db6-aca8-bbed78e40452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:21.042519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bb30e-fb57-4db6-aca8-bbed78e40452 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:21.059667 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1136/2256] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:21.065171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cc21c-30a4-46d6-a3d2-9b7c4bb32200 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:21.077793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cc21c-30a4-46d6-a3d2-9b7c4bb32200 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:21.083836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2aa8c-4396-4527-b261-bf49e9987e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:21.084840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2aa8c-4396-4527-b261-bf49e9987e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:21.085197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2aa8c-4396-4527-b261-bf49e9987e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:21.085590 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2aa8c-4396-4527-b261-bf49e9987e65 admin admin] PATH_INFO: `/v3/projects/c4ef300fdf4f4dc3825cdfc5a993cbe4/users/28e2c20fd2684369b941a5798c3602a1/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:21.107358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2aa8c-4396-4527-b261-bf49e9987e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4ef300fdf4f4dc3825cdfc5a993cbe4, user_id=28e2c20fd2684369b941a5798c3602a1, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:21.110412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2aa8c-4396-4527-b261-bf49e9987e65 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:21.298822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1121/2257] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:21 2022] PUT /identity/v3/projects/c4ef300fdf4f4dc3825cdfc5a993cbe4/users/28e2c20fd2684369b941a5798c3602a1/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:49:21.310370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1c0a-e70d-4558-b9d2-10ad7a8beb48 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:21.310921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1c0a-e70d-4558-b9d2-10ad7a8beb48 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:21.311250 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1c0a-e70d-4558-b9d2-10ad7a8beb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:21.362485 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6f6d1c0a-e70d-4558-b9d2-10ad7a8beb48 None None] MFA Rules not processed for user `28e2c20fd2684369b941a5798c3602a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:21.364558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6f6d1c0a-e70d-4558-b9d2-10ad7a8beb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:21.467961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1137/2258] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:21.475849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2aa8c-4396-4527-b261-bf49e9987e65 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:21.601214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2aa8c-4396-4527-b261-bf49e9987e65 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:21.609422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497805b-d925-41c3-b6dd-46d2607b327d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:21.610581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497805b-d925-41c3-b6dd-46d2607b327d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:21.611056 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497805b-d925-41c3-b6dd-46d2607b327d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:21.611526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497805b-d925-41c3-b6dd-46d2607b327d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:21.612717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497805b-d925-41c3-b6dd-46d2607b327d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:21.617049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497805b-d925-41c3-b6dd-46d2607b327d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:21.732507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1122/2259] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:49:21 2022] POST /identity/v3/projects => generated 385 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:21.738016 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d1c0a-e70d-4558-b9d2-10ad7a8beb48 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:21.749838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d1c0a-e70d-4558-b9d2-10ad7a8beb48 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:21.810186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:21.817821 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c926ac-2164-40d6-a9d7-948a35222d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:21.818494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c926ac-2164-40d6-a9d7-948a35222d89 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:21.818776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c926ac-2164-40d6-a9d7-948a35222d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:21.819044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c926ac-2164-40d6-a9d7-948a35222d89 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:21.819673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c926ac-2164-40d6-a9d7-948a35222d89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:21.821629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c926ac-2164-40d6-a9d7-948a35222d89 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:22.022844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1138/2260] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:49:21 2022] POST /identity/v3/users => generated 299 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:22.027077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497805b-d925-41c3-b6dd-46d2607b327d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:22.050362 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497805b-d925-41c3-b6dd-46d2607b327d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:22.061308 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1af7e-133a-46cd-8a7b-513017b907e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:22.062443 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1af7e-133a-46cd-8a7b-513017b907e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:22.062885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1af7e-133a-46cd-8a7b-513017b907e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:22.063343 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1af7e-133a-46cd-8a7b-513017b907e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:22.064638 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1af7e-133a-46cd-8a7b-513017b907e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:22.067666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1af7e-133a-46cd-8a7b-513017b907e7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:22.097439 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1123/2261] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:49:22.099221 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c926ac-2164-40d6-a9d7-948a35222d89 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:22.115253 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c926ac-2164-40d6-a9d7-948a35222d89 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:22.121778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd2aa1-fd9a-4646-9eef-21f7cc04ad83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:22.122797 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd2aa1-fd9a-4646-9eef-21f7cc04ad83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:22.123073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd2aa1-fd9a-4646-9eef-21f7cc04ad83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:22.123341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd2aa1-fd9a-4646-9eef-21f7cc04ad83 admin admin] PATH_INFO: `/v3/projects/22813bc8c7b3478988d92f7c7773b7a6/users/1d544e85495f4470a34d07cbb72bb1a4/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:22.151802 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd2aa1-fd9a-4646-9eef-21f7cc04ad83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22813bc8c7b3478988d92f7c7773b7a6, user_id=1d544e85495f4470a34d07cbb72bb1a4, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:22.156040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd2aa1-fd9a-4646-9eef-21f7cc04ad83 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:22.385850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1139/2262] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:22 2022] PUT /identity/v3/projects/22813bc8c7b3478988d92f7c7773b7a6/users/1d544e85495f4470a34d07cbb72bb1a4/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:22.390539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1af7e-133a-46cd-8a7b-513017b907e7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:22.425488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1af7e-133a-46cd-8a7b-513017b907e7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:22.501519 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:22.607115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71caba59-7d1c-4416-b1fc-a4bcfa78adf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:22.608738 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caba59-7d1c-4416-b1fc-a4bcfa78adf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:22.609416 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caba59-7d1c-4416-b1fc-a4bcfa78adf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:22.610034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caba59-7d1c-4416-b1fc-a4bcfa78adf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:22.614464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71caba59-7d1c-4416-b1fc-a4bcfa78adf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:22.615051 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71caba59-7d1c-4416-b1fc-a4bcfa78adf7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:22.643089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1124/2263] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:22.648805 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd2aa1-fd9a-4646-9eef-21f7cc04ad83 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:22.664640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd2aa1-fd9a-4646-9eef-21f7cc04ad83 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:22.683254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bd616-3273-4510-bfa3-e6e8189975cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:22.683254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bd616-3273-4510-bfa3-e6e8189975cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:22.683254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bd616-3273-4510-bfa3-e6e8189975cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:22.683879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bd616-3273-4510-bfa3-e6e8189975cd admin admin] PATH_INFO: `/v3/projects/22813bc8c7b3478988d92f7c7773b7a6/users/1d544e85495f4470a34d07cbb72bb1a4/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:22.686761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bd616-3273-4510-bfa3-e6e8189975cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=22813bc8c7b3478988d92f7c7773b7a6, user_id=1d544e85495f4470a34d07cbb72bb1a4, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:22.698452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bd616-3273-4510-bfa3-e6e8189975cd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:22.738634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1140/2264] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:22 2022] PUT /identity/v3/projects/22813bc8c7b3478988d92f7c7773b7a6/users/1d544e85495f4470a34d07cbb72bb1a4/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:22.747459 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8959fc-972e-438e-b92f-c0edb00757e0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:22.747778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8959fc-972e-438e-b92f-c0edb00757e0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:22.748015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8959fc-972e-438e-b92f-c0edb00757e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:22.813715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-df8959fc-972e-438e-b92f-c0edb00757e0 None None] MFA Rules not processed for user `1d544e85495f4470a34d07cbb72bb1a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:22.822424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-df8959fc-972e-438e-b92f-c0edb00757e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:22.980692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1125/2265] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:23.623559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494511d-93f2-47e1-8efb-d0b8736af698 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:23.624311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494511d-93f2-47e1-8efb-d0b8736af698 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:23.624537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494511d-93f2-47e1-8efb-d0b8736af698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:23.664388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4494511d-93f2-47e1-8efb-d0b8736af698 None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:23.666536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4494511d-93f2-47e1-8efb-d0b8736af698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:23.688867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8959fc-972e-438e-b92f-c0edb00757e0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:23.693851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-df8959fc-972e-438e-b92f-c0edb00757e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:23.791847 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1141/2266] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:49:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:23.802217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8959fc-972e-438e-b92f-c0edb00757e0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:23.816523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccad751-5f00-4a4f-9301-82f6b7065254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:23.816523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccad751-5f00-4a4f-9301-82f6b7065254 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:23.816523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccad751-5f00-4a4f-9301-82f6b7065254 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:23.816523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccad751-5f00-4a4f-9301-82f6b7065254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:23.818330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494511d-93f2-47e1-8efb-d0b8736af698 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:23.818330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4494511d-93f2-47e1-8efb-d0b8736af698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:23.820527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccad751-5f00-4a4f-9301-82f6b7065254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:23.825299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccad751-5f00-4a4f-9301-82f6b7065254 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:23.840244 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1126/2267] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:23 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:49:24.010381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494511d-93f2-47e1-8efb-d0b8736af698 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:24.031584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204310f5-4e74-48d4-8f7a-cbf26374903c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:24.032244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204310f5-4e74-48d4-8f7a-cbf26374903c service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:24.032537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204310f5-4e74-48d4-8f7a-cbf26374903c service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:24.032783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204310f5-4e74-48d4-8f7a-cbf26374903c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:24.058859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204310f5-4e74-48d4-8f7a-cbf26374903c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:24.079269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204310f5-4e74-48d4-8f7a-cbf26374903c service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:24.110710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1142/2268] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:49:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:49:26.782117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccad751-5f00-4a4f-9301-82f6b7065254 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:26.797341 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cccad751-5f00-4a4f-9301-82f6b7065254 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:26.853693 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccad751-5f00-4a4f-9301-82f6b7065254 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:26.870826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307ce94-cc7e-4df7-b5aa-77160a28f5ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:26.874464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ce94-cc7e-4df7-b5aa-77160a28f5ec service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:26.875368 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ce94-cc7e-4df7-b5aa-77160a28f5ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:26.876107 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ce94-cc7e-4df7-b5aa-77160a28f5ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:27.079631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:27.093717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307ce94-cc7e-4df7-b5aa-77160a28f5ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:27.098788 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307ce94-cc7e-4df7-b5aa-77160a28f5ec service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:27.146540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1127/2269] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:49:32.689954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a141f0-0950-417e-83c2-f6eaa57d6dd0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:32.692303 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a141f0-0950-417e-83c2-f6eaa57d6dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:32.693136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a141f0-0950-417e-83c2-f6eaa57d6dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:32.734544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-70a141f0-0950-417e-83c2-f6eaa57d6dd0 None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:32.737156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-70a141f0-0950-417e-83c2-f6eaa57d6dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:32.768057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1143/2270] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:49:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:32.815065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307ce94-cc7e-4df7-b5aa-77160a28f5ec service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:32.857220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307ce94-cc7e-4df7-b5aa-77160a28f5ec service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:32.873062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff758105-2e36-4dd6-9f45-2882ee472ff9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:32.874227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff758105-2e36-4dd6-9f45-2882ee472ff9 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:32.874733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff758105-2e36-4dd6-9f45-2882ee472ff9 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:32.875427 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff758105-2e36-4dd6-9f45-2882ee472ff9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:32.897615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff758105-2e36-4dd6-9f45-2882ee472ff9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:32.900271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff758105-2e36-4dd6-9f45-2882ee472ff9 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:32.913548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1128/2271] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:49:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 08:49:34.307249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab4b1c-a97f-4fc2-bd73-3ddcc12a2553 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:34.309794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab4b1c-a97f-4fc2-bd73-3ddcc12a2553 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:34.309794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab4b1c-a97f-4fc2-bd73-3ddcc12a2553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:34.406995 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c4ab4b1c-a97f-4fc2-bd73-3ddcc12a2553 None None] MFA Rules not processed for user `2aea4a0e0ec74c538735b890fb774b14`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:34.412740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c4ab4b1c-a97f-4fc2-bd73-3ddcc12a2553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:34.590416 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:34.757938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1144/2272] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:38.888914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff758105-2e36-4dd6-9f45-2882ee472ff9 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:38.901361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff758105-2e36-4dd6-9f45-2882ee472ff9 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:38.907920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd46f9-2369-4401-a666-494b35b68866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:38.908251 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd46f9-2369-4401-a666-494b35b68866 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:38.908700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd46f9-2369-4401-a666-494b35b68866 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:38.909146 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd46f9-2369-4401-a666-494b35b68866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:39.025365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd46f9-2369-4401-a666-494b35b68866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:39.027059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd46f9-2369-4401-a666-494b35b68866 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:39.072626 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1129/2273] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:49:43.301933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab4b1c-a97f-4fc2-bd73-3ddcc12a2553 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:43.304768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c4ab4b1c-a97f-4fc2-bd73-3ddcc12a2553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:43.544135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab4b1c-a97f-4fc2-bd73-3ddcc12a2553 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:43.563169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:43.567941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:43.567941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:43.567941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] PATH_INFO: `/v3/users/f2fdef783c244114a4161e3a6489b672` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:43.569501 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2fdef783c244114a4161e3a6489b672)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:43.578973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:44.381601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2fdef783c244114a4161e3a6489b672'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:44.388026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2fdef783c244114a4161e3a6489b672'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:44.537706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2fdef783c244114a4161e3a6489b672'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:44.557489 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2fdef783c244114a4161e3a6489b672'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:44.604148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2fdef783c244114a4161e3a6489b672'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:44.662540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1145/2274] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:49:43 2022] DELETE /identity/v3/users/f2fdef783c244114a4161e3a6489b672 => generated 0 bytes in 1328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:45.296689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd46f9-2369-4401-a666-494b35b68866 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:45.338950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd46f9-2369-4401-a666-494b35b68866 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:45.648960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:45.649983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:45.650478 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:45.650933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] PATH_INFO: `/v3/projects/3a58131788a94091a90239807e46e789` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:45.653158 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a58131788a94091a90239807e46e789)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:45.656708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:45.954342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a58131788a94091a90239807e46e789', 'request_id': 'req-3f3dd0bc-1475-433b-bca0-895773c13769'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:46.224830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a58131788a94091a90239807e46e789', 'request_id': 'req-3f3dd0bc-1475-433b-bca0-895773c13769'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:46.240876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a58131788a94091a90239807e46e789', 'request_id': 'req-3f3dd0bc-1475-433b-bca0-895773c13769'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:46.286929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] The token cache is being invalidate because project 3a58131788a94091a90239807e46e789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:49:46.287399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:46.288255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:46.295613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1130/2275] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:49:45 2022] DELETE /identity/v3/projects/3a58131788a94091a90239807e46e789 => generated 0 bytes in 995 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:46.296907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:46.299276 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:46.316572 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68764e2-9c98-4d37-82d1-9da25789c407 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:46.455327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:46.456081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:46.456942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:46.457218 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] PATH_INFO: `/v3/users/cb20b5ed2e1848559356fb9ebd79a623` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:46.458441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb20b5ed2e1848559356fb9ebd79a623)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:46.460395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:46.462793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:46.972664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb20b5ed2e1848559356fb9ebd79a623'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:46.973410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb20b5ed2e1848559356fb9ebd79a623'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:47.173146 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb20b5ed2e1848559356fb9ebd79a623'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:47.210177 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb20b5ed2e1848559356fb9ebd79a623'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:47.260094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb20b5ed2e1848559356fb9ebd79a623'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:47.275555 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1146/2276] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:49:46 2022] DELETE /identity/v3/users/cb20b5ed2e1848559356fb9ebd79a623 => generated 0 bytes in 979 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:47.386538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:47.389699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:47.407345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3dd0bc-1475-433b-bca0-895773c13769 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:47.424731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:47.662338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573692-4314-4585-9f6f-edb8b25065c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:47.665361 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573692-4314-4585-9f6f-edb8b25065c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:47.665841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573692-4314-4585-9f6f-edb8b25065c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:47.669715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573692-4314-4585-9f6f-edb8b25065c5 admin admin] PATH_INFO: `/v3/users/a518d39c799842c9bd30e38fc61b9cee` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:47.675587 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f573692-4314-4585-9f6f-edb8b25065c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a518d39c799842c9bd30e38fc61b9cee)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:47.678890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f573692-4314-4585-9f6f-edb8b25065c5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:48.165332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:48.167503 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:48.211333 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b4058-9954-4355-b665-47f7e8a061f2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:48.226547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4f573692-4314-4585-9f6f-edb8b25065c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a518d39c799842c9bd30e38fc61b9cee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:48.230185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61eb92d-e764-4492-9e01-6a12db68a517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:48.231634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61eb92d-e764-4492-9e01-6a12db68a517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:48.252759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61eb92d-e764-4492-9e01-6a12db68a517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:48.259984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61eb92d-e764-4492-9e01-6a12db68a517 admin admin] PATH_INFO: `/v3/projects/7f64599e7ba94e4daa8e5389a167ae49` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:48.264956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61eb92d-e764-4492-9e01-6a12db68a517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f64599e7ba94e4daa8e5389a167ae49)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:48.274856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61eb92d-e764-4492-9e01-6a12db68a517 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:48.286113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4f573692-4314-4585-9f6f-edb8b25065c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a518d39c799842c9bd30e38fc61b9cee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:48.395367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4f573692-4314-4585-9f6f-edb8b25065c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a518d39c799842c9bd30e38fc61b9cee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:48.396950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4f573692-4314-4585-9f6f-edb8b25065c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a518d39c799842c9bd30e38fc61b9cee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:48.428807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4f573692-4314-4585-9f6f-edb8b25065c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a518d39c799842c9bd30e38fc61b9cee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:48.446240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1131/2277] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:49:47 2022] DELETE /identity/v3/users/a518d39c799842c9bd30e38fc61b9cee => generated 0 bytes in 1058 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:48.448261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e7368-befd-4f0b-9601-221061171af2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:48.448709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e7368-befd-4f0b-9601-221061171af2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:48.449265 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e7368-befd-4f0b-9601-221061171af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:48.475331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b45e7368-befd-4f0b-9601-221061171af2 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:48.476974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b45e7368-befd-4f0b-9601-221061171af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:48.507575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d61eb92d-e764-4492-9e01-6a12db68a517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f64599e7ba94e4daa8e5389a167ae49', 'request_id': 'req-d61eb92d-e764-4492-9e01-6a12db68a517'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:48.603708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d61eb92d-e764-4492-9e01-6a12db68a517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f64599e7ba94e4daa8e5389a167ae49', 'request_id': 'req-d61eb92d-e764-4492-9e01-6a12db68a517'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:48.641063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d61eb92d-e764-4492-9e01-6a12db68a517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f64599e7ba94e4daa8e5389a167ae49', 'request_id': 'req-d61eb92d-e764-4492-9e01-6a12db68a517'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:48.656971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1132/2278] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:48.671340 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e7368-befd-4f0b-9601-221061171af2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:48.672937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b45e7368-befd-4f0b-9601-221061171af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:48.697816 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d61eb92d-e764-4492-9e01-6a12db68a517 admin admin] The token cache is being invalidate because project 7f64599e7ba94e4daa8e5389a167ae49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:49:48.698106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d61eb92d-e764-4492-9e01-6a12db68a517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:48.698925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d61eb92d-e764-4492-9e01-6a12db68a517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:48.705673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1147/2279] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:49:48 2022] DELETE /identity/v3/projects/7f64599e7ba94e4daa8e5389a167ae49 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:49:48.709531 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30112bc-1915-4110-8112-f1c7ee537b60 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:48.713498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30112bc-1915-4110-8112-f1c7ee537b60 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:48.713498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30112bc-1915-4110-8112-f1c7ee537b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:48.736055 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:48.770736 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d30112bc-1915-4110-8112-f1c7ee537b60 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:48.771877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d30112bc-1915-4110-8112-f1c7ee537b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:48.795052 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1148/2280] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:48.796557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f778c-2699-4b43-abac-31fd3c8e5258 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:48.796883 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f778c-2699-4b43-abac-31fd3c8e5258 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:48.797161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f778c-2699-4b43-abac-31fd3c8e5258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:48.803897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e7368-befd-4f0b-9601-221061171af2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:48.820507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74155d-cab6-4e70-a075-dbfe72ae9493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:48.834053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74155d-cab6-4e70-a075-dbfe72ae9493 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:48.834053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74155d-cab6-4e70-a075-dbfe72ae9493 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:48.834053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74155d-cab6-4e70-a075-dbfe72ae9493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:48.834571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3f2f778c-2699-4b43-abac-31fd3c8e5258 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:48.834571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3f2f778c-2699-4b43-abac-31fd3c8e5258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:48.834848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0e74155d-cab6-4e70-a075-dbfe72ae9493 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:48.844653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1149/2281] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:48.845771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f778c-2699-4b43-abac-31fd3c8e5258 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:48.857770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f778c-2699-4b43-abac-31fd3c8e5258 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:48.881708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74155d-cab6-4e70-a075-dbfe72ae9493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:48.884485 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74155d-cab6-4e70-a075-dbfe72ae9493 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:48.884719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841794a-78b9-415f-88ca-c5322d6d0579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:48.885290 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841794a-78b9-415f-88ca-c5322d6d0579 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:48.885675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841794a-78b9-415f-88ca-c5322d6d0579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:48.885964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841794a-78b9-415f-88ca-c5322d6d0579 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:48.886974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841794a-78b9-415f-88ca-c5322d6d0579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:49:48.887758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841794a-78b9-415f-88ca-c5322d6d0579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:48.893655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841794a-78b9-415f-88ca-c5322d6d0579 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:48.913104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1133/2282] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:49:48.950458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1150/2283] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:49:48.952543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74155d-cab6-4e70-a075-dbfe72ae9493 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:48.972197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74155d-cab6-4e70-a075-dbfe72ae9493 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:48.977821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e15690-6778-405f-baeb-f41c40ad20a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:48.978848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e15690-6778-405f-baeb-f41c40ad20a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:48.980970 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e15690-6778-405f-baeb-f41c40ad20a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:48.981413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e15690-6778-405f-baeb-f41c40ad20a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:48.982350 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e15690-6778-405f-baeb-f41c40ad20a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:48.984901 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e15690-6778-405f-baeb-f41c40ad20a3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:49.114092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1134/2284] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:49:48 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:49.121547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841794a-78b9-415f-88ca-c5322d6d0579 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:49.144256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841794a-78b9-415f-88ca-c5322d6d0579 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:49.231170 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:49.287101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82250e-be1e-4544-aed7-1e8f44181ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:49.287964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82250e-be1e-4544-aed7-1e8f44181ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:49.288252 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82250e-be1e-4544-aed7-1e8f44181ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:49.288838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82250e-be1e-4544-aed7-1e8f44181ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:49.289510 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82250e-be1e-4544-aed7-1e8f44181ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:49.294652 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82250e-be1e-4544-aed7-1e8f44181ec7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:49.474692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1151/2285] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:49:49 2022] POST /identity/v3/users => generated 292 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:49.479296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e15690-6778-405f-baeb-f41c40ad20a3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:49.499796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e15690-6778-405f-baeb-f41c40ad20a3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:49.531452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b26067-04f5-4965-a75e-06b0ec8f4d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:49.532708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b26067-04f5-4965-a75e-06b0ec8f4d4f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:49.533261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b26067-04f5-4965-a75e-06b0ec8f4d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:49.533822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b26067-04f5-4965-a75e-06b0ec8f4d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:49.535975 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b26067-04f5-4965-a75e-06b0ec8f4d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:49.539113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b26067-04f5-4965-a75e-06b0ec8f4d4f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:49.584612 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1135/2286] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:49:49.586202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82250e-be1e-4544-aed7-1e8f44181ec7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:49.601340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82250e-be1e-4544-aed7-1e8f44181ec7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:49.619427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bde78-a32d-4b44-a208-26f734e9d365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:49.620090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bde78-a32d-4b44-a208-26f734e9d365 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:49.620874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bde78-a32d-4b44-a208-26f734e9d365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:49.620874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bde78-a32d-4b44-a208-26f734e9d365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:49.621305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bde78-a32d-4b44-a208-26f734e9d365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:49.622990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bde78-a32d-4b44-a208-26f734e9d365 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:49.667546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1152/2287] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:49.669785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b26067-04f5-4965-a75e-06b0ec8f4d4f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:49.699998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b26067-04f5-4965-a75e-06b0ec8f4d4f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:49.709310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce29250-dd9a-477c-8535-5cac9563ea95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:49.710333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce29250-dd9a-477c-8535-5cac9563ea95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:49.710726 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce29250-dd9a-477c-8535-5cac9563ea95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:49.711138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce29250-dd9a-477c-8535-5cac9563ea95 admin admin] PATH_INFO: `/v3/projects/e32e2f0e291d4236bd8aa9281d15e517/users/d80e3065938b45b89ace4f62e9082bc4/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:49.761227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce29250-dd9a-477c-8535-5cac9563ea95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e32e2f0e291d4236bd8aa9281d15e517, user_id=d80e3065938b45b89ace4f62e9082bc4, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:49.764905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce29250-dd9a-477c-8535-5cac9563ea95 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:49.864704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1136/2288] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:49 2022] PUT /identity/v3/projects/e32e2f0e291d4236bd8aa9281d15e517/users/d80e3065938b45b89ace4f62e9082bc4/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:49.876703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd2d16-254f-4260-ae63-ed09fa09dc52 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:49.876983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd2d16-254f-4260-ae63-ed09fa09dc52 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:49.877241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd2d16-254f-4260-ae63-ed09fa09dc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:49.955502 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-71dd2d16-254f-4260-ae63-ed09fa09dc52 None None] MFA Rules not processed for user `d80e3065938b45b89ace4f62e9082bc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:49.957346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-71dd2d16-254f-4260-ae63-ed09fa09dc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:50.062691 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce29250-dd9a-477c-8535-5cac9563ea95 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:50.067175 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dce29250-dd9a-477c-8535-5cac9563ea95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:50.201399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1153/2289] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:50.225680 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd2d16-254f-4260-ae63-ed09fa09dc52 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:50.232459 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-71dd2d16-254f-4260-ae63-ed09fa09dc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:50.321758 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce29250-dd9a-477c-8535-5cac9563ea95 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:50.343057 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:50.343920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:50.345651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:50.345651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] PATH_INFO: `/v3/projects/f4391e7d8e4044e49433c2fbc1f1b94f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:50.351230 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4391e7d8e4044e49433c2fbc1f1b94f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:50.354282 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:50.397830 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd2d16-254f-4260-ae63-ed09fa09dc52 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:50.409993 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc7c57-9bd1-4bf7-81bd-b576b3f7b00d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:50.410971 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc7c57-9bd1-4bf7-81bd-b576b3f7b00d service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:50.411424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc7c57-9bd1-4bf7-81bd-b576b3f7b00d service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:50.411795 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc7c57-9bd1-4bf7-81bd-b576b3f7b00d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:50.438580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc7c57-9bd1-4bf7-81bd-b576b3f7b00d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:50.438580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc7c57-9bd1-4bf7-81bd-b576b3f7b00d service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:50.461640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1154/2290] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:49:50.569726 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4391e7d8e4044e49433c2fbc1f1b94f', 'request_id': 'req-f0a27350-d8f9-4df5-aa99-71efe53428bf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:50.663043 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4391e7d8e4044e49433c2fbc1f1b94f', 'request_id': 'req-f0a27350-d8f9-4df5-aa99-71efe53428bf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:50.672394 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4391e7d8e4044e49433c2fbc1f1b94f', 'request_id': 'req-f0a27350-d8f9-4df5-aa99-71efe53428bf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:50.699377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] The token cache is being invalidate because project f4391e7d8e4044e49433c2fbc1f1b94f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:49:50.699701 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:50.705339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:50.710121 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1137/2291] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:49:50 2022] DELETE /identity/v3/projects/f4391e7d8e4044e49433c2fbc1f1b94f => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:50.716129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc7c57-9bd1-4bf7-81bd-b576b3f7b00d service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:50.739655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc7c57-9bd1-4bf7-81bd-b576b3f7b00d service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:50.860778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:50.933046 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:50.933861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:50.934190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:50.935258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] PATH_INFO: `/v3/users/28e2c20fd2684369b941a5798c3602a1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:50.940027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e2c20fd2684369b941a5798c3602a1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:50.944478 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:51.351533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e2c20fd2684369b941a5798c3602a1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:51.353507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e2c20fd2684369b941a5798c3602a1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:51.511820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:51.514313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:51.529235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:51.555278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e2c20fd2684369b941a5798c3602a1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:51.579135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e2c20fd2684369b941a5798c3602a1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:51.604994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e2c20fd2684369b941a5798c3602a1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:51.628907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1155/2292] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:49:50 2022] DELETE /identity/v3/users/28e2c20fd2684369b941a5798c3602a1 => generated 0 bytes in 909 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:51.676775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a27350-d8f9-4df5-aa99-71efe53428bf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:51.783042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feadd09-9a98-403d-a30f-912d807fd1cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:51.783841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feadd09-9a98-403d-a30f-912d807fd1cf service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:51.784905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feadd09-9a98-403d-a30f-912d807fd1cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:51.786062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feadd09-9a98-403d-a30f-912d807fd1cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:51.788337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9feadd09-9a98-403d-a30f-912d807fd1cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:52.002541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feadd09-9a98-403d-a30f-912d807fd1cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:52.005582 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feadd09-9a98-403d-a30f-912d807fd1cf service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:52.035094 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1138/2293] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:49:53.358689 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:53.361675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:53.464984 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feadd09-9a98-403d-a30f-912d807fd1cf service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:53.475450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9feadd09-9a98-403d-a30f-912d807fd1cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:53.535202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff135ce6-0c63-4131-b906-2a7c01677321 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:53.541242 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:53.541923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:53.542234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:53.543671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] PATH_INFO: `/v3/projects/c4ef300fdf4f4dc3825cdfc5a993cbe4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:53.544971 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4ef300fdf4f4dc3825cdfc5a993cbe4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:53.548167 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:53.756130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4ef300fdf4f4dc3825cdfc5a993cbe4', 'request_id': 'req-4a908b59-41f7-4c86-8d06-4a6ba252d37d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:53.871286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feadd09-9a98-403d-a30f-912d807fd1cf service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:53.915903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4ef300fdf4f4dc3825cdfc5a993cbe4', 'request_id': 'req-4a908b59-41f7-4c86-8d06-4a6ba252d37d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:53.919435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:53.945845 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4ef300fdf4f4dc3825cdfc5a993cbe4', 'request_id': 'req-4a908b59-41f7-4c86-8d06-4a6ba252d37d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:53.959355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:54.007542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] The token cache is being invalidate because project c4ef300fdf4f4dc3825cdfc5a993cbe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:49:54.013034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:54.013034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:54.013034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1156/2294] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:49:53 2022] DELETE /identity/v3/projects/c4ef300fdf4f4dc3825cdfc5a993cbe4 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:54.018507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:54.043213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:54.186238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922be515-cbde-48d5-9bd7-b501ffd2bc4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d03db170fb6460794e32669456bcdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:54.187272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922be515-cbde-48d5-9bd7-b501ffd2bc4d service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:54.187825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922be515-cbde-48d5-9bd7-b501ffd2bc4d service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:54.188375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922be515-cbde-48d5-9bd7-b501ffd2bc4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:54.192705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-922be515-cbde-48d5-9bd7-b501ffd2bc4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:54.227406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922be515-cbde-48d5-9bd7-b501ffd2bc4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:54.231069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922be515-cbde-48d5-9bd7-b501ffd2bc4d service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:54.256210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1139/2295] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 792 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:49:54.396614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a908b59-41f7-4c86-8d06-4a6ba252d37d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:54.419879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0fd24-bd0a-4802-a194-3914955bd90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:54.424322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0fd24-bd0a-4802-a194-3914955bd90a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:54.427546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0fd24-bd0a-4802-a194-3914955bd90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:54.427546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0fd24-bd0a-4802-a194-3914955bd90a admin admin] PATH_INFO: `/v3/users/1d544e85495f4470a34d07cbb72bb1a4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:54.431620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0fd24-bd0a-4802-a194-3914955bd90a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d544e85495f4470a34d07cbb72bb1a4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:54.441807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0fd24-bd0a-4802-a194-3914955bd90a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:54.741758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d7d0fd24-bd0a-4802-a194-3914955bd90a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d544e85495f4470a34d07cbb72bb1a4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:54.743034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d7d0fd24-bd0a-4802-a194-3914955bd90a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d544e85495f4470a34d07cbb72bb1a4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:54.816896 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d7d0fd24-bd0a-4802-a194-3914955bd90a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d544e85495f4470a34d07cbb72bb1a4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:54.829719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d7d0fd24-bd0a-4802-a194-3914955bd90a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d544e85495f4470a34d07cbb72bb1a4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:54.850492 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d7d0fd24-bd0a-4802-a194-3914955bd90a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d544e85495f4470a34d07cbb72bb1a4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:54.873481 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1157/2296] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:49:54 2022] DELETE /identity/v3/users/1d544e85495f4470a34d07cbb72bb1a4 => generated 0 bytes in 857 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:55.741794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922be515-cbde-48d5-9bd7-b501ffd2bc4d service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:55.758687 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922be515-cbde-48d5-9bd7-b501ffd2bc4d service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:55.875337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da1ad2-60d8-49c9-8757-33fdd601e858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:55.875337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da1ad2-60d8-49c9-8757-33fdd601e858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:55.875337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da1ad2-60d8-49c9-8757-33fdd601e858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:55.876615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da1ad2-60d8-49c9-8757-33fdd601e858 admin admin] PATH_INFO: `/v3/projects/22813bc8c7b3478988d92f7c7773b7a6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:55.877449 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da1ad2-60d8-49c9-8757-33fdd601e858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22813bc8c7b3478988d92f7c7773b7a6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:55.881504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da1ad2-60d8-49c9-8757-33fdd601e858 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:56.097353 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-01da1ad2-60d8-49c9-8757-33fdd601e858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22813bc8c7b3478988d92f7c7773b7a6', 'request_id': 'req-01da1ad2-60d8-49c9-8757-33fdd601e858'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:56.195363 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-01da1ad2-60d8-49c9-8757-33fdd601e858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22813bc8c7b3478988d92f7c7773b7a6', 'request_id': 'req-01da1ad2-60d8-49c9-8757-33fdd601e858'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:56.213073 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-01da1ad2-60d8-49c9-8757-33fdd601e858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22813bc8c7b3478988d92f7c7773b7a6', 'request_id': 'req-01da1ad2-60d8-49c9-8757-33fdd601e858'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:56.279077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-01da1ad2-60d8-49c9-8757-33fdd601e858 admin admin] The token cache is being invalidate because project 22813bc8c7b3478988d92f7c7773b7a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:49:56.279554 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-01da1ad2-60d8-49c9-8757-33fdd601e858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:56.283611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-01da1ad2-60d8-49c9-8757-33fdd601e858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:49:56.286067 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1140/2297] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:49:55 2022] DELETE /identity/v3/projects/22813bc8c7b3478988d92f7c7773b7a6 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:49:56.306500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab86e4e-b39f-4309-bd03-ffda6954d280 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:56.312133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab86e4e-b39f-4309-bd03-ffda6954d280 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:56.312133 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab86e4e-b39f-4309-bd03-ffda6954d280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:56.359488 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7ab86e4e-b39f-4309-bd03-ffda6954d280 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:56.360961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7ab86e4e-b39f-4309-bd03-ffda6954d280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:56.488003 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1158/2298] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:56.490258 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bbd86-f6cf-4d55-a067-34795fe3b475 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:56.490696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bbd86-f6cf-4d55-a067-34795fe3b475 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:56.491072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bbd86-f6cf-4d55-a067-34795fe3b475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:56.525014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-665bbd86-f6cf-4d55-a067-34795fe3b475 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:56.526502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-665bbd86-f6cf-4d55-a067-34795fe3b475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:56.541688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1141/2299] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:49:56.546018 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab86e4e-b39f-4309-bd03-ffda6954d280 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:56.560763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab86e4e-b39f-4309-bd03-ffda6954d280 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:56.577127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8300b8-9fbc-4d9b-a366-4f0199296636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:56.579217 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8300b8-9fbc-4d9b-a366-4f0199296636 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:56.580311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8300b8-9fbc-4d9b-a366-4f0199296636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:56.581879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8300b8-9fbc-4d9b-a366-4f0199296636 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:56.584912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8300b8-9fbc-4d9b-a366-4f0199296636 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:49:56.585627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8300b8-9fbc-4d9b-a366-4f0199296636 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:56.590006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8300b8-9fbc-4d9b-a366-4f0199296636 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:56.622798 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1159/2300] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:49:56.624372 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bbd86-f6cf-4d55-a067-34795fe3b475 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:56.636561 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bbd86-f6cf-4d55-a067-34795fe3b475 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:56.643945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3412d-3342-4b7d-98d1-0818bba5d2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:56.646031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3412d-3342-4b7d-98d1-0818bba5d2a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:56.646031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3412d-3342-4b7d-98d1-0818bba5d2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:56.646031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3412d-3342-4b7d-98d1-0818bba5d2a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:56.646367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3412d-3342-4b7d-98d1-0818bba5d2a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:56.648314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3412d-3342-4b7d-98d1-0818bba5d2a6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:56.822931 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1142/2301] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:49:56 2022] POST /identity/v3/projects => generated 409 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:56.826444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8300b8-9fbc-4d9b-a366-4f0199296636 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:56.832665 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:56.838269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8300b8-9fbc-4d9b-a366-4f0199296636 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:56.981022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13080cd-4753-4c9a-b3c4-35ca9cc5e4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:56.981022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13080cd-4753-4c9a-b3c4-35ca9cc5e4bd admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:56.981022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13080cd-4753-4c9a-b3c4-35ca9cc5e4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:56.981022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13080cd-4753-4c9a-b3c4-35ca9cc5e4bd admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:56.982602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13080cd-4753-4c9a-b3c4-35ca9cc5e4bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:56.989259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13080cd-4753-4c9a-b3c4-35ca9cc5e4bd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:57.188826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1160/2302] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:49:56 2022] POST /identity/v3/users => generated 305 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:57.198332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3412d-3342-4b7d-98d1-0818bba5d2a6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:57.212219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:57.226942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3412d-3342-4b7d-98d1-0818bba5d2a6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:57.242946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dddb1a-f246-416c-8f14-40bcf46549e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:57.247473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dddb1a-f246-416c-8f14-40bcf46549e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:57.247473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dddb1a-f246-416c-8f14-40bcf46549e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:57.247473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dddb1a-f246-416c-8f14-40bcf46549e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:57.247473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dddb1a-f246-416c-8f14-40bcf46549e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:57.251318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dddb1a-f246-416c-8f14-40bcf46549e1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:57.294201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1143/2303] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:57.300253 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13080cd-4753-4c9a-b3c4-35ca9cc5e4bd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:57.320477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13080cd-4753-4c9a-b3c4-35ca9cc5e4bd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:57.334052 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010367c-d870-4976-8e7e-9e7de945aeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:57.335382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010367c-d870-4976-8e7e-9e7de945aeb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:57.335689 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010367c-d870-4976-8e7e-9e7de945aeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:57.337139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010367c-d870-4976-8e7e-9e7de945aeb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:57.339366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010367c-d870-4976-8e7e-9e7de945aeb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:57.342820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010367c-d870-4976-8e7e-9e7de945aeb9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:57.391790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1161/2304] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:57.396706 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dddb1a-f246-416c-8f14-40bcf46549e1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:57.413164 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dddb1a-f246-416c-8f14-40bcf46549e1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:57.434974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169b97f-b381-498e-8b9b-b5e7bc53a46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:57.436129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169b97f-b381-498e-8b9b-b5e7bc53a46b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:57.436578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169b97f-b381-498e-8b9b-b5e7bc53a46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:57.437021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169b97f-b381-498e-8b9b-b5e7bc53a46b admin admin] PATH_INFO: `/v3/projects/4eb08f9e70d047f0b978c27fefd4f94f/users/878080668d5f40babdbba77eb5c15f13/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:57.493845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169b97f-b381-498e-8b9b-b5e7bc53a46b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eb08f9e70d047f0b978c27fefd4f94f, user_id=878080668d5f40babdbba77eb5c15f13, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:57.508380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169b97f-b381-498e-8b9b-b5e7bc53a46b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:57.670357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1144/2305] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:57 2022] PUT /identity/v3/projects/4eb08f9e70d047f0b978c27fefd4f94f/users/878080668d5f40babdbba77eb5c15f13/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:57.681830 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84332aa-4886-475b-8104-3dfdab8ea16a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:57.686400 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84332aa-4886-475b-8104-3dfdab8ea16a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:57.686400 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84332aa-4886-475b-8104-3dfdab8ea16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:57.771202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e84332aa-4886-475b-8104-3dfdab8ea16a None None] MFA Rules not processed for user `878080668d5f40babdbba77eb5c15f13`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:57.772582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e84332aa-4886-475b-8104-3dfdab8ea16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:57.936889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1162/2306] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:57 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:57.944127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169b97f-b381-498e-8b9b-b5e7bc53a46b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:58.185197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169b97f-b381-498e-8b9b-b5e7bc53a46b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:58.198703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69876738-234f-4e6c-af09-1086a541d005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:58.199579 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69876738-234f-4e6c-af09-1086a541d005 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:58.199895 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69876738-234f-4e6c-af09-1086a541d005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:58.200283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69876738-234f-4e6c-af09-1086a541d005 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:58.201060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69876738-234f-4e6c-af09-1086a541d005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:58.205731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69876738-234f-4e6c-af09-1086a541d005 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:58.391885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1145/2307] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:49:57 2022] POST /identity/v3/projects => generated 409 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:49:58.397649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84332aa-4886-475b-8104-3dfdab8ea16a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:58.421808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84332aa-4886-475b-8104-3dfdab8ea16a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:58.799073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591299d-d2d7-45ec-8be7-8d7aa0fe5f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:58.802540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591299d-d2d7-45ec-8be7-8d7aa0fe5f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:58.802540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591299d-d2d7-45ec-8be7-8d7aa0fe5f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:58.802540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591299d-d2d7-45ec-8be7-8d7aa0fe5f75 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:58.802540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591299d-d2d7-45ec-8be7-8d7aa0fe5f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:58.830698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591299d-d2d7-45ec-8be7-8d7aa0fe5f75 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:58.973745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:58.982447 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1163/2308] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:49:58 2022] POST /identity/v3/users => generated 311 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:49:58.985764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69876738-234f-4e6c-af09-1086a541d005 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:59.012891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69876738-234f-4e6c-af09-1086a541d005 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:59.023320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64e304-ceb7-40c9-81ee-8f11a9d60c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:59.024145 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64e304-ceb7-40c9-81ee-8f11a9d60c21 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:59.024761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64e304-ceb7-40c9-81ee-8f11a9d60c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:59.025420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64e304-ceb7-40c9-81ee-8f11a9d60c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:59.029634 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64e304-ceb7-40c9-81ee-8f11a9d60c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:59.032214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64e304-ceb7-40c9-81ee-8f11a9d60c21 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:59.063061 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1146/2309] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:49:59.067995 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591299d-d2d7-45ec-8be7-8d7aa0fe5f75 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:59.094828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591299d-d2d7-45ec-8be7-8d7aa0fe5f75 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:59.107247 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9896f-1a40-43bb-b471-835f10e8488c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:59.108774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9896f-1a40-43bb-b471-835f10e8488c admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:59.109169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9896f-1a40-43bb-b471-835f10e8488c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:59.109665 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9896f-1a40-43bb-b471-835f10e8488c admin admin] PATH_INFO: `/v3/projects/fc2c5ff315bf455eb365328b59c7531a/users/bdcf538940a247b597c044d350dc6fe2/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:59.130867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9896f-1a40-43bb-b471-835f10e8488c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2c5ff315bf455eb365328b59c7531a, user_id=bdcf538940a247b597c044d350dc6fe2, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:59.134316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9896f-1a40-43bb-b471-835f10e8488c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:59.248558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1164/2310] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:59 2022] PUT /identity/v3/projects/fc2c5ff315bf455eb365328b59c7531a/users/bdcf538940a247b597c044d350dc6fe2/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:59.255310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64e304-ceb7-40c9-81ee-8f11a9d60c21 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:59.289396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64e304-ceb7-40c9-81ee-8f11a9d60c21 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:59.557515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7991d56-228c-4370-ae20-02e8d75c8c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:59.558262 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7991d56-228c-4370-ae20-02e8d75c8c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:59.558548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7991d56-228c-4370-ae20-02e8d75c8c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:59.558817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7991d56-228c-4370-ae20-02e8d75c8c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:59.559473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7991d56-228c-4370-ae20-02e8d75c8c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:59.563182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7991d56-228c-4370-ae20-02e8d75c8c03 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:59.591347 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1147/2311] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:49:59.595932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9896f-1a40-43bb-b471-835f10e8488c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:59.616619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9896f-1a40-43bb-b471-835f10e8488c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:49:59.626038 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e8f58-4c7e-4e11-8a66-86f811aacc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:49:59.627102 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e8f58-4c7e-4e11-8a66-86f811aacc26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:59.628121 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e8f58-4c7e-4e11-8a66-86f811aacc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:59.628627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e8f58-4c7e-4e11-8a66-86f811aacc26 admin admin] PATH_INFO: `/v3/projects/fc2c5ff315bf455eb365328b59c7531a/users/bdcf538940a247b597c044d350dc6fe2/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:59.631147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e8f58-4c7e-4e11-8a66-86f811aacc26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2c5ff315bf455eb365328b59c7531a, user_id=bdcf538940a247b597c044d350dc6fe2, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:49:59.634402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e8f58-4c7e-4e11-8a66-86f811aacc26 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:49:59.651787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1165/2312] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:49:59 2022] PUT /identity/v3/projects/fc2c5ff315bf455eb365328b59c7531a/users/bdcf538940a247b597c044d350dc6fe2/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:49:59.663722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24059f59-b6ea-4aeb-adfe-d76c1754cf66 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:49:59.664215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24059f59-b6ea-4aeb-adfe-d76c1754cf66 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:49:59.664416 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24059f59-b6ea-4aeb-adfe-d76c1754cf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:49:59.714586 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-24059f59-b6ea-4aeb-adfe-d76c1754cf66 None None] MFA Rules not processed for user `bdcf538940a247b597c044d350dc6fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:49:59.715755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-24059f59-b6ea-4aeb-adfe-d76c1754cf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:49:59.775162 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:49:59.895125 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1148/2313] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:49:59 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:49:59.914886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e8f58-4c7e-4e11-8a66-86f811aacc26 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:49:59.918700 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-903e8f58-4c7e-4e11-8a66-86f811aacc26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:00.062866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e8f58-4c7e-4e11-8a66-86f811aacc26 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:00.072990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa405af-d255-43d0-9489-f2901f3b97cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:00.073979 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa405af-d255-43d0-9489-f2901f3b97cb service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:00.074468 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa405af-d255-43d0-9489-f2901f3b97cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:00.078547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa405af-d255-43d0-9489-f2901f3b97cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:00.096859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa405af-d255-43d0-9489-f2901f3b97cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:00.099423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa405af-d255-43d0-9489-f2901f3b97cb service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:00.118508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1166/2314] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:49:59 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:00.726359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24059f59-b6ea-4aeb-adfe-d76c1754cf66 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:00.728735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-24059f59-b6ea-4aeb-adfe-d76c1754cf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:00.759602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24059f59-b6ea-4aeb-adfe-d76c1754cf66 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:00.774393 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32ea9d-19e4-4f56-9273-c69805f51cba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:00.775110 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32ea9d-19e4-4f56-9273-c69805f51cba service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:00.775466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32ea9d-19e4-4f56-9273-c69805f51cba service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:00.775815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32ea9d-19e4-4f56-9273-c69805f51cba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:01.014096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32ea9d-19e4-4f56-9273-c69805f51cba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:01.018353 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32ea9d-19e4-4f56-9273-c69805f51cba service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:01.065403 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1149/2315] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:00 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:03.194849 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa405af-d255-43d0-9489-f2901f3b97cb service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:03.200487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0aa405af-d255-43d0-9489-f2901f3b97cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:03.421537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa405af-d255-43d0-9489-f2901f3b97cb service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:03.439422 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:03.440655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:03.444113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:03.444487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] PATH_INFO: `/v3/users/d80e3065938b45b89ace4f62e9082bc4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:03.447590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d80e3065938b45b89ace4f62e9082bc4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:03.452314 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:03.530101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908de7b-c240-4a93-bf9c-35d4586ad00e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:03.531131 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908de7b-c240-4a93-bf9c-35d4586ad00e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:03.531637 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908de7b-c240-4a93-bf9c-35d4586ad00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:03.560235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f908de7b-c240-4a93-bf9c-35d4586ad00e None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:03.562075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f908de7b-c240-4a93-bf9c-35d4586ad00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:03.729727 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1150/2316] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:50:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:50:03.744836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908de7b-c240-4a93-bf9c-35d4586ad00e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:03.747174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f908de7b-c240-4a93-bf9c-35d4586ad00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:03.897213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd80e3065938b45b89ace4f62e9082bc4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:03.899566 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd80e3065938b45b89ace4f62e9082bc4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:03.956229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908de7b-c240-4a93-bf9c-35d4586ad00e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:03.974060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc70762-676e-4465-a185-64c7c0ad3d19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:03.974836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70762-676e-4465-a185-64c7c0ad3d19 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:03.975672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70762-676e-4465-a185-64c7c0ad3d19 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:03.975972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70762-676e-4465-a185-64c7c0ad3d19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:03.978278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd80e3065938b45b89ace4f62e9082bc4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:03.980278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9cc70762-676e-4465-a185-64c7c0ad3d19 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:04.012719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd80e3065938b45b89ace4f62e9082bc4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:04.013007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc70762-676e-4465-a185-64c7c0ad3d19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:04.016732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc70762-676e-4465-a185-64c7c0ad3d19 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:04.030049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:04.037591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:04.043113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1151/2317] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:50:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:50:04.086766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd80e3065938b45b89ace4f62e9082bc4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:04.106097 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1167/2318] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:50:03 2022] DELETE /identity/v3/users/d80e3065938b45b89ace4f62e9082bc4 => generated 0 bytes in 907 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:04.108404 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faa54b-8abb-40bb-a5e5-4a143cd53487 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:04.108404 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faa54b-8abb-40bb-a5e5-4a143cd53487 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:04.108404 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faa54b-8abb-40bb-a5e5-4a143cd53487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:04.146940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c4faa54b-8abb-40bb-a5e5-4a143cd53487 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:04.148182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c4faa54b-8abb-40bb-a5e5-4a143cd53487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:04.319229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1152/2319] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:50:04.330924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:04.331678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:04.448894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b17dea-a9ab-4833-bd26-5a70613823bd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:04.452758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc996057-c336-4d53-9060-8f7885dd4421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:04.456968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc996057-c336-4d53-9060-8f7885dd4421 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:04.456968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc996057-c336-4d53-9060-8f7885dd4421 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:04.456968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc996057-c336-4d53-9060-8f7885dd4421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:04.524747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc996057-c336-4d53-9060-8f7885dd4421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:04.526344 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc996057-c336-4d53-9060-8f7885dd4421 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:04.547731 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1168/2320] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:05.303317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4faa54b-8abb-40bb-a5e5-4a143cd53487 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:05.305886 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c4faa54b-8abb-40bb-a5e5-4a143cd53487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:05.358197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4faa54b-8abb-40bb-a5e5-4a143cd53487 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:05.376625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ec33d-8410-4a3f-8dfb-919b44e068ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:05.379844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ec33d-8410-4a3f-8dfb-919b44e068ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:05.382214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ec33d-8410-4a3f-8dfb-919b44e068ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:05.382613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ec33d-8410-4a3f-8dfb-919b44e068ac admin admin] PATH_INFO: `/v3/projects/e32e2f0e291d4236bd8aa9281d15e517` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:05.387079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ec33d-8410-4a3f-8dfb-919b44e068ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=e32e2f0e291d4236bd8aa9281d15e517)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:05.396096 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ec33d-8410-4a3f-8dfb-919b44e068ac admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:05.473625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161a880-e3fe-4226-a696-b78013a39a7c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:05.474129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161a880-e3fe-4226-a696-b78013a39a7c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:05.474306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161a880-e3fe-4226-a696-b78013a39a7c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:05.475744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1169/2321] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:50:05.483312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a29a34-0bb7-4769-8245-584c9a9ea34d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:05.483873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a29a34-0bb7-4769-8245-584c9a9ea34d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:05.484395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a29a34-0bb7-4769-8245-584c9a9ea34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:05.516567 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-53a29a34-0bb7-4769-8245-584c9a9ea34d None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:05.519702 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-53a29a34-0bb7-4769-8245-584c9a9ea34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:05.630367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-951ec33d-8410-4a3f-8dfb-919b44e068ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e32e2f0e291d4236bd8aa9281d15e517', 'request_id': 'req-951ec33d-8410-4a3f-8dfb-919b44e068ac'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:05.656032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1170/2322] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:50:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:50:05.697112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a29a34-0bb7-4769-8245-584c9a9ea34d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:05.698798 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-53a29a34-0bb7-4769-8245-584c9a9ea34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:05.734170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-951ec33d-8410-4a3f-8dfb-919b44e068ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e32e2f0e291d4236bd8aa9281d15e517', 'request_id': 'req-951ec33d-8410-4a3f-8dfb-919b44e068ac'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:05.759802 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-951ec33d-8410-4a3f-8dfb-919b44e068ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e32e2f0e291d4236bd8aa9281d15e517', 'request_id': 'req-951ec33d-8410-4a3f-8dfb-919b44e068ac'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:05.807960 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a29a34-0bb7-4769-8245-584c9a9ea34d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:05.809520 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-951ec33d-8410-4a3f-8dfb-919b44e068ac admin admin] The token cache is being invalidate because project e32e2f0e291d4236bd8aa9281d15e517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:50:05.809897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-951ec33d-8410-4a3f-8dfb-919b44e068ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:05.813568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-951ec33d-8410-4a3f-8dfb-919b44e068ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:05.816142 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1153/2323] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:50:05 2022] DELETE /identity/v3/projects/e32e2f0e291d4236bd8aa9281d15e517 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:05.821412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c7060-1f8e-4220-a4d7-9571c85043d7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:05.821843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c7060-1f8e-4220-a4d7-9571c85043d7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:05.822126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c7060-1f8e-4220-a4d7-9571c85043d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:05.880462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-627c7060-1f8e-4220-a4d7-9571c85043d7 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:05.884340 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-627c7060-1f8e-4220-a4d7-9571c85043d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:05.903928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:05.950617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c5431-6e2e-4dc7-8680-057b64ff8436 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b86978df61245ae88fead154193fcd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:05.951258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c5431-6e2e-4dc7-8680-057b64ff8436 service glance] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:05.951539 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c5431-6e2e-4dc7-8680-057b64ff8436 service glance] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:05.951812 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c5431-6e2e-4dc7-8680-057b64ff8436 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:05.953898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-154c5431-6e2e-4dc7-8680-057b64ff8436 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:06.084567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1154/2324] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:06.092509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb73d2-016f-41ac-8ed8-fdcf1a4600d0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:06.092775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb73d2-016f-41ac-8ed8-fdcf1a4600d0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:06.093051 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb73d2-016f-41ac-8ed8-fdcf1a4600d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:06.140560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1ffb73d2-016f-41ac-8ed8-fdcf1a4600d0 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:06.140560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1ffb73d2-016f-41ac-8ed8-fdcf1a4600d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:06.142200 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c5431-6e2e-4dc7-8680-057b64ff8436 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:06.147994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c5431-6e2e-4dc7-8680-057b64ff8436 service glance] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:06.166392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1155/2325] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:50:06.172984 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb73d2-016f-41ac-8ed8-fdcf1a4600d0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:06.185503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb73d2-016f-41ac-8ed8-fdcf1a4600d0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:06.190396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1171/2326] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:50:06.192376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a90ab-3b70-4ff5-b338-c364b2225ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:06.192981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a90ab-3b70-4ff5-b338-c364b2225ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:06.193261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a90ab-3b70-4ff5-b338-c364b2225ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:06.193536 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a90ab-3b70-4ff5-b338-c364b2225ce0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:06.194870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a90ab-3b70-4ff5-b338-c364b2225ce0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:50:06.195193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a90ab-3b70-4ff5-b338-c364b2225ce0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:06.211364 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a90ab-3b70-4ff5-b338-c364b2225ce0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:06.249717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c5431-6e2e-4dc7-8680-057b64ff8436 service glance] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:06.250277 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1156/2327] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:50:06.269567 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c5431-6e2e-4dc7-8680-057b64ff8436 service glance] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:06.275096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07126eb2-bfd4-4aa3-86d9-b92f7d6564c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:06.275734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07126eb2-bfd4-4aa3-86d9-b92f7d6564c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:06.276015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07126eb2-bfd4-4aa3-86d9-b92f7d6564c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:06.276287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07126eb2-bfd4-4aa3-86d9-b92f7d6564c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:06.276939 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07126eb2-bfd4-4aa3-86d9-b92f7d6564c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:06.279031 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07126eb2-bfd4-4aa3-86d9-b92f7d6564c7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:06.402090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1172/2328] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:50:06 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:06.406290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a90ab-3b70-4ff5-b338-c364b2225ce0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:06.417056 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:06.425455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a90ab-3b70-4ff5-b338-c364b2225ce0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:06.518909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b952b6-79d4-4128-a4f9-b338c031f191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:06.519662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b952b6-79d4-4128-a4f9-b338c031f191 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:06.520035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b952b6-79d4-4128-a4f9-b338c031f191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:06.520407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b952b6-79d4-4128-a4f9-b338c031f191 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:06.521454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b952b6-79d4-4128-a4f9-b338c031f191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:06.524062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b952b6-79d4-4128-a4f9-b338c031f191 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:06.562696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431498b9-fabf-4068-a88a-39ee439b15a1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:06.563393 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431498b9-fabf-4068-a88a-39ee439b15a1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:06.564093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431498b9-fabf-4068-a88a-39ee439b15a1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:06.566082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1173/2329] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:50:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 08:50:06.571145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ca83f-1aa2-46a7-a490-0e39572c5ecd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:06.571430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ca83f-1aa2-46a7-a490-0e39572c5ecd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:06.571697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ca83f-1aa2-46a7-a490-0e39572c5ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:06.594326 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d35ca83f-1aa2-46a7-a490-0e39572c5ecd None None] MFA Rules not processed for user `92867e15dcf649adad78bf7a29a996f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:06.595581 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d35ca83f-1aa2-46a7-a490-0e39572c5ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:06.639602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1157/2330] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:50:06 2022] POST /identity/v3/users => generated 284 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:06.644332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b952b6-79d4-4128-a4f9-b338c031f191 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:06.668839 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b952b6-79d4-4128-a4f9-b338c031f191 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:06.680921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630ad36-67a0-40d2-9184-e62994830fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:06.681544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630ad36-67a0-40d2-9184-e62994830fec admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:06.682844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630ad36-67a0-40d2-9184-e62994830fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:06.690761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630ad36-67a0-40d2-9184-e62994830fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:06.691640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630ad36-67a0-40d2-9184-e62994830fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:06.693119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1174/2331] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:50:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:50:06.693792 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630ad36-67a0-40d2-9184-e62994830fec admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:06.713983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ca83f-1aa2-46a7-a490-0e39572c5ecd None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:06.714708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1158/2332] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:50:06.715752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630ad36-67a0-40d2-9184-e62994830fec admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:06.717368 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d35ca83f-1aa2-46a7-a490-0e39572c5ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:06.730462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630ad36-67a0-40d2-9184-e62994830fec admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:06.736380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ca83f-1aa2-46a7-a490-0e39572c5ecd None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:06.751704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cd72d-39a8-4799-a7b3-0ae03294877c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:06.751704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cd72d-39a8-4799-a7b3-0ae03294877c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:06.751704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cd72d-39a8-4799-a7b3-0ae03294877c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:06.751704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cd72d-39a8-4799-a7b3-0ae03294877c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:06.752131 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cd72d-39a8-4799-a7b3-0ae03294877c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:06.752211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cab550-6e71-48ef-9912-9bea49f3c120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:06.752211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cab550-6e71-48ef-9912-9bea49f3c120 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:06.752211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cab550-6e71-48ef-9912-9bea49f3c120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:06.752211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cab550-6e71-48ef-9912-9bea49f3c120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:06.752593 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cd72d-39a8-4799-a7b3-0ae03294877c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:06.762061 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cab550-6e71-48ef-9912-9bea49f3c120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:06.764163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cab550-6e71-48ef-9912-9bea49f3c120 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:06.767642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1159/2333] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:50:06.767642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05536eae-10ea-48de-800b-50e2ca975d84 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:06.767642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05536eae-10ea-48de-800b-50e2ca975d84 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:06.767642 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05536eae-10ea-48de-800b-50e2ca975d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:06.780465 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1175/2334] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:50:06.782896 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cab550-6e71-48ef-9912-9bea49f3c120 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:06.798301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cab550-6e71-48ef-9912-9bea49f3c120 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:06.807892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b5d1f-88a2-4d84-a8bd-c2cf36d81f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:06.808766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b5d1f-88a2-4d84-a8bd-c2cf36d81f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:06.809167 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b5d1f-88a2-4d84-a8bd-c2cf36d81f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:06.809564 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b5d1f-88a2-4d84-a8bd-c2cf36d81f70 admin admin] PATH_INFO: `/v3/projects/2e2f0654cfd143b282c0ae4dc98a7de4/users/4d4d8919181a46a69ae08b4a80e3bf63/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:06.823533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-05536eae-10ea-48de-800b-50e2ca975d84 None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:06.825515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-05536eae-10ea-48de-800b-50e2ca975d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:06.841665 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b5d1f-88a2-4d84-a8bd-c2cf36d81f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2f0654cfd143b282c0ae4dc98a7de4, user_id=4d4d8919181a46a69ae08b4a80e3bf63, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:06.850829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b5d1f-88a2-4d84-a8bd-c2cf36d81f70 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:06.927542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1160/2335] 10.222.0.44 () {58 vars in 967 bytes} [Sun Sep 11 08:50:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:06.928837 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c7d5d-7513-4131-a3f1-df27b4bce90e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:06.929137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c7d5d-7513-4131-a3f1-df27b4bce90e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:06.929414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c7d5d-7513-4131-a3f1-df27b4bce90e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:06.930597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1161/2336] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:50:06.938874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2e47d-0f13-4c7f-a240-26dc786a15e5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:06.939153 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2e47d-0f13-4c7f-a240-26dc786a15e5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:06.939419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2e47d-0f13-4c7f-a240-26dc786a15e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:06.954757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1176/2337] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:50:06 2022] PUT /identity/v3/projects/2e2f0654cfd143b282c0ae4dc98a7de4/users/4d4d8919181a46a69ae08b4a80e3bf63/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:06.956859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b5d1f-88a2-4d84-a8bd-c2cf36d81f70 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:06.959649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f05b5d1f-88a2-4d84-a8bd-c2cf36d81f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:06.971953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-69c2e47d-0f13-4c7f-a240-26dc786a15e5 None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:06.973199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-69c2e47d-0f13-4c7f-a240-26dc786a15e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:07.014051 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:07.058194 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1162/2338] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:50:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:07.061433 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f9911-b8f8-4142-bd95-854a8b6766ac None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:07.061829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f9911-b8f8-4142-bd95-854a8b6766ac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:07.062286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f9911-b8f8-4142-bd95-854a8b6766ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:07.066800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b5d1f-88a2-4d84-a8bd-c2cf36d81f70 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:07.074902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d42ac-e13d-42ba-adaf-96d9844f4617 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:07.075615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d42ac-e13d-42ba-adaf-96d9844f4617 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:07.075977 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d42ac-e13d-42ba-adaf-96d9844f4617 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:07.076429 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d42ac-e13d-42ba-adaf-96d9844f4617 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:07.108979 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8d1f9911-b8f8-4142-bd95-854a8b6766ac None None] MFA Rules not processed for user `4d4d8919181a46a69ae08b4a80e3bf63`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:07.110235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8d1f9911-b8f8-4142-bd95-854a8b6766ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:07.155132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d42ac-e13d-42ba-adaf-96d9844f4617 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:07.156740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d42ac-e13d-42ba-adaf-96d9844f4617 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:07.194947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1177/2339] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:50:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:50:07.194947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d42ac-e13d-42ba-adaf-96d9844f4617 service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:07.211195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1163/2340] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:07.228582 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f9911-b8f8-4142-bd95-854a8b6766ac None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:07.230807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8d1f9911-b8f8-4142-bd95-854a8b6766ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:07.284552 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d42ac-e13d-42ba-adaf-96d9844f4617 service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:07.288837 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e308f-0d4c-47bf-b443-48220008e4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:07.289800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e308f-0d4c-47bf-b443-48220008e4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:07.290378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e308f-0d4c-47bf-b443-48220008e4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:07.290718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e308f-0d4c-47bf-b443-48220008e4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:07.303355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e308f-0d4c-47bf-b443-48220008e4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:07.304946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e308f-0d4c-47bf-b443-48220008e4a1 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:07.321483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1178/2341] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:07.330022 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f9911-b8f8-4142-bd95-854a8b6766ac None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:07.335665 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeed4ce-7b14-446f-bf48-2a1b5a06b744 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b86978df61245ae88fead154193fcd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:07.336753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeed4ce-7b14-446f-bf48-2a1b5a06b744 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:07.337259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeed4ce-7b14-446f-bf48-2a1b5a06b744 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:07.337747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeed4ce-7b14-446f-bf48-2a1b5a06b744 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:07.354037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeed4ce-7b14-446f-bf48-2a1b5a06b744 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:07.355919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeed4ce-7b14-446f-bf48-2a1b5a06b744 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:07.374300 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1164/2342] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:50:07.377029 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:08.102565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728249ad-8b5f-41d2-8daa-cfce6c895202 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:08.103557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728249ad-8b5f-41d2-8daa-cfce6c895202 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:08.104121 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728249ad-8b5f-41d2-8daa-cfce6c895202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:08.145354 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-728249ad-8b5f-41d2-8daa-cfce6c895202 None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:08.156815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-728249ad-8b5f-41d2-8daa-cfce6c895202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:08.181433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1179/2343] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:50:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:08.196256 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeed4ce-7b14-446f-bf48-2a1b5a06b744 service glance] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:08.216027 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeed4ce-7b14-446f-bf48-2a1b5a06b744 service glance] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:08.233435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc25f0-b4ce-47f6-a887-1b95fabcc88b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:08.234540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc25f0-b4ce-47f6-a887-1b95fabcc88b service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:08.238166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc25f0-b4ce-47f6-a887-1b95fabcc88b service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:08.239094 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc25f0-b4ce-47f6-a887-1b95fabcc88b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:08.259906 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc25f0-b4ce-47f6-a887-1b95fabcc88b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:08.263379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc25f0-b4ce-47f6-a887-1b95fabcc88b service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:08.308971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1165/2344] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:50:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 08:50:08.775062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fe8d2-7b6d-4a0d-96bb-b03712893d71 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:08.775990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fe8d2-7b6d-4a0d-96bb-b03712893d71 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:08.776770 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fe8d2-7b6d-4a0d-96bb-b03712893d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:08.824780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-107fe8d2-7b6d-4a0d-96bb-b03712893d71 None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:08.828306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-107fe8d2-7b6d-4a0d-96bb-b03712893d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:08.847107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1180/2345] 10.222.0.44 () {58 vars in 967 bytes} [Sun Sep 11 08:50:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:08.897927 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc25f0-b4ce-47f6-a887-1b95fabcc88b service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:08.912489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc25f0-b4ce-47f6-a887-1b95fabcc88b service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:08.933482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f67692-aef2-4487-a6b8-34e0b5af1929 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:08.934415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f67692-aef2-4487-a6b8-34e0b5af1929 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:08.935149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f67692-aef2-4487-a6b8-34e0b5af1929 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:08.935149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f67692-aef2-4487-a6b8-34e0b5af1929 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:08.953779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f67692-aef2-4487-a6b8-34e0b5af1929 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:08.957276 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f67692-aef2-4487-a6b8-34e0b5af1929 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:08.982242 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1166/2346] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:50:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:50:09.354586 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dc8cb-b713-4f95-afe1-b187a8235233 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:09.355415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dc8cb-b713-4f95-afe1-b187a8235233 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:09.355415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dc8cb-b713-4f95-afe1-b187a8235233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:09.400764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d77dc8cb-b713-4f95-afe1-b187a8235233 None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:09.400764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d77dc8cb-b713-4f95-afe1-b187a8235233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:09.422709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1181/2347] 10.222.0.44 () {58 vars in 967 bytes} [Sun Sep 11 08:50:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:09.450936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f67692-aef2-4487-a6b8-34e0b5af1929 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:09.470043 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f67692-aef2-4487-a6b8-34e0b5af1929 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:09.482065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cfa9c-7ad9-4053-b1b7-ef20cf9eb12e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:09.483482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cfa9c-7ad9-4053-b1b7-ef20cf9eb12e service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:09.484016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cfa9c-7ad9-4053-b1b7-ef20cf9eb12e service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:09.484496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cfa9c-7ad9-4053-b1b7-ef20cf9eb12e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:09.510739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cfa9c-7ad9-4053-b1b7-ef20cf9eb12e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:09.510739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cfa9c-7ad9-4053-b1b7-ef20cf9eb12e service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:09.522057 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1167/2348] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:50:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 08:50:13.095924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77dc8cb-b713-4f95-afe1-b187a8235233 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:13.099974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d77dc8cb-b713-4f95-afe1-b187a8235233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:13.230027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:13.262521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77dc8cb-b713-4f95-afe1-b187a8235233 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:13.275188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d96bb3-979a-4ec1-8308-6d389ce65305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:13.276671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d96bb3-979a-4ec1-8308-6d389ce65305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:13.277297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d96bb3-979a-4ec1-8308-6d389ce65305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:13.279062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d96bb3-979a-4ec1-8308-6d389ce65305 admin admin] PATH_INFO: `/v3/users/7c2338046a594f9eb4f2fef030f1b82b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:13.283403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d96bb3-979a-4ec1-8308-6d389ce65305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2338046a594f9eb4f2fef030f1b82b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:13.289283 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d96bb3-979a-4ec1-8308-6d389ce65305 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:13.426820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cfa9c-7ad9-4053-b1b7-ef20cf9eb12e service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:13.430743 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b66cfa9c-7ad9-4053-b1b7-ef20cf9eb12e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:13.479862 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cfa9c-7ad9-4053-b1b7-ef20cf9eb12e service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:13.492402 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f0f4e-06eb-4f3f-b758-2892d0b94a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:13.496440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f0f4e-06eb-4f3f-b758-2892d0b94a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:13.496440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f0f4e-06eb-4f3f-b758-2892d0b94a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:13.496440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f0f4e-06eb-4f3f-b758-2892d0b94a83 admin admin] PATH_INFO: `/v3/users/878080668d5f40babdbba77eb5c15f13` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:13.496440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f0f4e-06eb-4f3f-b758-2892d0b94a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=878080668d5f40babdbba77eb5c15f13)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:13.498728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f0f4e-06eb-4f3f-b758-2892d0b94a83 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:13.585215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d6d96bb3-979a-4ec1-8308-6d389ce65305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2338046a594f9eb4f2fef030f1b82b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:13.585959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d6d96bb3-979a-4ec1-8308-6d389ce65305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2338046a594f9eb4f2fef030f1b82b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:13.707968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d6d96bb3-979a-4ec1-8308-6d389ce65305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2338046a594f9eb4f2fef030f1b82b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:13.716828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d6d96bb3-979a-4ec1-8308-6d389ce65305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2338046a594f9eb4f2fef030f1b82b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:13.737443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d6d96bb3-979a-4ec1-8308-6d389ce65305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2338046a594f9eb4f2fef030f1b82b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:13.752996 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1182/2349] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:50:13 2022] DELETE /identity/v3/users/7c2338046a594f9eb4f2fef030f1b82b => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:13.752996 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c43f6b-67f1-467a-a196-adfb18e0ba41 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:13.752996 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c43f6b-67f1-467a-a196-adfb18e0ba41 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:13.752996 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c43f6b-67f1-467a-a196-adfb18e0ba41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:13.777907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-955f0f4e-06eb-4f3f-b758-2892d0b94a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '878080668d5f40babdbba77eb5c15f13'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:13.788553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-21c43f6b-67f1-467a-a196-adfb18e0ba41 None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:13.790588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-21c43f6b-67f1-467a-a196-adfb18e0ba41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:13.792146 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:13.809594 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-955f0f4e-06eb-4f3f-b758-2892d0b94a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '878080668d5f40babdbba77eb5c15f13'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:13.894009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1183/2350] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:50:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:13.894864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c43f6b-67f1-467a-a196-adfb18e0ba41 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:13.896577 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-21c43f6b-67f1-467a-a196-adfb18e0ba41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:13.900576 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-955f0f4e-06eb-4f3f-b758-2892d0b94a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '878080668d5f40babdbba77eb5c15f13'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:13.901972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-955f0f4e-06eb-4f3f-b758-2892d0b94a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '878080668d5f40babdbba77eb5c15f13'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:13.918987 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-955f0f4e-06eb-4f3f-b758-2892d0b94a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '878080668d5f40babdbba77eb5c15f13'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:13.935804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1168/2351] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:50:13 2022] DELETE /identity/v3/users/878080668d5f40babdbba77eb5c15f13 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:13.938075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075eb30-09bd-485f-81c5-51b990eb67eb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:13.938624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075eb30-09bd-485f-81c5-51b990eb67eb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:13.939103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075eb30-09bd-485f-81c5-51b990eb67eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:13.964320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5075eb30-09bd-485f-81c5-51b990eb67eb None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:13.965937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5075eb30-09bd-485f-81c5-51b990eb67eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:14.053871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c43f6b-67f1-467a-a196-adfb18e0ba41 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:14.075909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1169/2352] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:14.077348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075eb30-09bd-485f-81c5-51b990eb67eb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:14.079509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5075eb30-09bd-485f-81c5-51b990eb67eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:14.099030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:14.174313 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:14.174910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:14.175332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:14.175985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] PATH_INFO: `/v3/users/0b198ff6ed104068b38b993054617f9a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:14.178621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b198ff6ed104068b38b993054617f9a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:14.181205 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:14.257883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075eb30-09bd-485f-81c5-51b990eb67eb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:14.270782 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70952eed-7826-469a-a83f-a4c9af43a234 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:14.271835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70952eed-7826-469a-a83f-a4c9af43a234 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:14.272299 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70952eed-7826-469a-a83f-a4c9af43a234 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:14.272759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70952eed-7826-469a-a83f-a4c9af43a234 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:14.276181 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-70952eed-7826-469a-a83f-a4c9af43a234 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:14.358412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70952eed-7826-469a-a83f-a4c9af43a234 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:14.360102 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70952eed-7826-469a-a83f-a4c9af43a234 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:14.369843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1170/2353] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:50:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:50:14.371510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db6ad2-ccbd-43e9-8db8-8d7360cfb400 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:14.371842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db6ad2-ccbd-43e9-8db8-8d7360cfb400 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:14.372392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db6ad2-ccbd-43e9-8db8-8d7360cfb400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:14.422395 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b9db6ad2-ccbd-43e9-8db8-8d7360cfb400 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:14.423928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b9db6ad2-ccbd-43e9-8db8-8d7360cfb400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:14.434304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1171/2354] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:14.435264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db6ad2-ccbd-43e9-8db8-8d7360cfb400 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:14.436686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b9db6ad2-ccbd-43e9-8db8-8d7360cfb400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:14.479406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:14.509533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db6ad2-ccbd-43e9-8db8-8d7360cfb400 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:14.514847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfd30a-141e-47d6-94ee-31a34ffbc574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:14.515756 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfd30a-141e-47d6-94ee-31a34ffbc574 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:14.516156 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfd30a-141e-47d6-94ee-31a34ffbc574 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:14.516614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfd30a-141e-47d6-94ee-31a34ffbc574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:14.532489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cfd30a-141e-47d6-94ee-31a34ffbc574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:14.534886 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cfd30a-141e-47d6-94ee-31a34ffbc574 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:14.548033 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1172/2355] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:14.551161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfd30a-141e-47d6-94ee-31a34ffbc574 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:14.552749 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-17cfd30a-141e-47d6-94ee-31a34ffbc574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:14.570140 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfd30a-141e-47d6-94ee-31a34ffbc574 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:14.578005 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede972a-b0c5-45c1-bc6f-afc2a6950f2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:14.579457 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede972a-b0c5-45c1-bc6f-afc2a6950f2e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:14.580960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede972a-b0c5-45c1-bc6f-afc2a6950f2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:14.580960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede972a-b0c5-45c1-bc6f-afc2a6950f2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:14.607773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede972a-b0c5-45c1-bc6f-afc2a6950f2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:14.609479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede972a-b0c5-45c1-bc6f-afc2a6950f2e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:14.621432 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1173/2356] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:14.856921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b198ff6ed104068b38b993054617f9a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:14.858341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b198ff6ed104068b38b993054617f9a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:15.200772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b198ff6ed104068b38b993054617f9a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:15.213886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b198ff6ed104068b38b993054617f9a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:15.231478 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b198ff6ed104068b38b993054617f9a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:15.244791 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1184/2357] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:50:13 2022] DELETE /identity/v3/users/0b198ff6ed104068b38b993054617f9a => generated 0 bytes in 1346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:15.445753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede972a-b0c5-45c1-bc6f-afc2a6950f2e service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:15.448612 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fede972a-b0c5-45c1-bc6f-afc2a6950f2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:15.503174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede972a-b0c5-45c1-bc6f-afc2a6950f2e service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:15.672042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:15.674250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:15.674820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:15.675387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] PATH_INFO: `/v3/projects/baaaaf5f58244280b0e265aa43bc571b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:15.678597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baaaaf5f58244280b0e265aa43bc571b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:15.692208 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:15.708824 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:15.711542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:15.750752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc29ff-279a-47ba-950c-721e908cc73e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:15.769950 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62574d0-f527-438e-9baf-85e256fbab21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:15.771277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62574d0-f527-438e-9baf-85e256fbab21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:15.771738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62574d0-f527-438e-9baf-85e256fbab21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:15.772495 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62574d0-f527-438e-9baf-85e256fbab21 admin admin] PATH_INFO: `/v3/projects/ead98e6e89d840a0b99273f28429b4ba` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:15.775030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62574d0-f527-438e-9baf-85e256fbab21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ead98e6e89d840a0b99273f28429b4ba)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:15.777372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62574d0-f527-438e-9baf-85e256fbab21 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:15.890728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baaaaf5f58244280b0e265aa43bc571b', 'request_id': 'req-bd37593d-c37d-485c-8227-df3f8e8e69f9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:16.038457 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baaaaf5f58244280b0e265aa43bc571b', 'request_id': 'req-bd37593d-c37d-485c-8227-df3f8e8e69f9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:16.064712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baaaaf5f58244280b0e265aa43bc571b', 'request_id': 'req-bd37593d-c37d-485c-8227-df3f8e8e69f9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:16.129589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] The token cache is being invalidate because project baaaaf5f58244280b0e265aa43bc571b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:50:16.133135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a62574d0-f527-438e-9baf-85e256fbab21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ead98e6e89d840a0b99273f28429b4ba', 'request_id': 'req-a62574d0-f527-438e-9baf-85e256fbab21'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:16.140796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:16.142867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:16.149908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1174/2358] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:50:15 2022] DELETE /identity/v3/projects/baaaaf5f58244280b0e265aa43bc571b => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:16.153287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:16.159390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:16.327921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a62574d0-f527-438e-9baf-85e256fbab21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ead98e6e89d840a0b99273f28429b4ba', 'request_id': 'req-a62574d0-f527-438e-9baf-85e256fbab21'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:16.336951 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:16.344571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a62574d0-f527-438e-9baf-85e256fbab21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ead98e6e89d840a0b99273f28429b4ba', 'request_id': 'req-a62574d0-f527-438e-9baf-85e256fbab21'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:16.363943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a62574d0-f527-438e-9baf-85e256fbab21 admin admin] The token cache is being invalidate because project ead98e6e89d840a0b99273f28429b4ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:50:16.364567 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a62574d0-f527-438e-9baf-85e256fbab21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:16.365687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a62574d0-f527-438e-9baf-85e256fbab21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:16.368109 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1185/2359] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:50:15 2022] DELETE /identity/v3/projects/ead98e6e89d840a0b99273f28429b4ba => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:16.371482 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dc074-29f9-4db9-8859-ce8c773b4b87 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:16.372183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dc074-29f9-4db9-8859-ce8c773b4b87 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:16.372629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dc074-29f9-4db9-8859-ce8c773b4b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:16.460768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37593d-c37d-485c-8227-df3f8e8e69f9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:16.467135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cc4dc074-29f9-4db9-8859-ce8c773b4b87 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:16.469042 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cc4dc074-29f9-4db9-8859-ce8c773b4b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:16.533977 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:16.652035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1186/2360] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:16.653678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42098963-a623-4aaf-8d01-88f8ac043906 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:16.654194 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42098963-a623-4aaf-8d01-88f8ac043906 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:16.654674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42098963-a623-4aaf-8d01-88f8ac043906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:16.657251 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:16.658626 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:16.662018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:16.662018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] PATH_INFO: `/v3/projects/4eb08f9e70d047f0b978c27fefd4f94f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:16.662018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eb08f9e70d047f0b978c27fefd4f94f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:16.662713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:16.685199 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-42098963-a623-4aaf-8d01-88f8ac043906 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:16.687546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-42098963-a623-4aaf-8d01-88f8ac043906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:16.711225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1187/2361] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:16.712915 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc77fbe-0cef-46e0-8fad-2cde9d06f547 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:16.713388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc77fbe-0cef-46e0-8fad-2cde9d06f547 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:16.713662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc77fbe-0cef-46e0-8fad-2cde9d06f547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:16.742916 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-dcc77fbe-0cef-46e0-8fad-2cde9d06f547 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:16.745107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dcc77fbe-0cef-46e0-8fad-2cde9d06f547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:16.802198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1188/2362] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:16.804398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e260c-0db3-4e8c-89fe-d9a57e479882 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:16.804976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e260c-0db3-4e8c-89fe-d9a57e479882 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:16.805985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e260c-0db3-4e8c-89fe-d9a57e479882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:16.871732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-839e260c-0db3-4e8c-89fe-d9a57e479882 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:16.874658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-839e260c-0db3-4e8c-89fe-d9a57e479882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:16.897981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eb08f9e70d047f0b978c27fefd4f94f', 'request_id': 'req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:16.999763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eb08f9e70d047f0b978c27fefd4f94f', 'request_id': 'req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:17.014304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eb08f9e70d047f0b978c27fefd4f94f', 'request_id': 'req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:17.040889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] The token cache is being invalidate because project 4eb08f9e70d047f0b978c27fefd4f94f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:50:17.041403 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:17.043192 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:17.054206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1175/2363] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:50:16 2022] DELETE /identity/v3/projects/4eb08f9e70d047f0b978c27fefd4f94f => generated 0 bytes in 896 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:17.054206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:17.067153 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:17.088333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e42b8c-d7d6-4ea2-9c98-5190da6ac473 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:17.089230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1189/2364] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:17.100660 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e260c-0db3-4e8c-89fe-d9a57e479882 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:17.111424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-839e260c-0db3-4e8c-89fe-d9a57e479882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:17.121714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93553a1-b787-4006-b53d-626b09c19ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:17.122520 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93553a1-b787-4006-b53d-626b09c19ebb admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:17.122808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93553a1-b787-4006-b53d-626b09c19ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:17.123088 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93553a1-b787-4006-b53d-626b09c19ebb admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:17.123745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93553a1-b787-4006-b53d-626b09c19ebb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:50:17.124046 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93553a1-b787-4006-b53d-626b09c19ebb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:17.125869 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93553a1-b787-4006-b53d-626b09c19ebb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:17.144705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e260c-0db3-4e8c-89fe-d9a57e479882 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:17.159425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:17.160164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:17.160439 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:17.160620 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] PATH_INFO: `/v3/users/bdcf538940a247b597c044d350dc6fe2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:17.164476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdcf538940a247b597c044d350dc6fe2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:17.165672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1176/2365] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:50:17.166897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:17.167142 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93553a1-b787-4006-b53d-626b09c19ebb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:17.195411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93553a1-b787-4006-b53d-626b09c19ebb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:17.227892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5dd22-64f6-4ec5-80bf-8c48e0804243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:17.228572 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5dd22-64f6-4ec5-80bf-8c48e0804243 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:17.232538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5dd22-64f6-4ec5-80bf-8c48e0804243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:17.232538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5dd22-64f6-4ec5-80bf-8c48e0804243 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:17.232538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5dd22-64f6-4ec5-80bf-8c48e0804243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:50:17.232538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5dd22-64f6-4ec5-80bf-8c48e0804243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:17.232538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5dd22-64f6-4ec5-80bf-8c48e0804243 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:17.253613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1177/2366] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:50:17.253613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5dd22-64f6-4ec5-80bf-8c48e0804243 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:17.282357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5dd22-64f6-4ec5-80bf-8c48e0804243 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:17.290744 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de7cea-7a66-4e1e-baef-ad8fee0668f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:17.291462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de7cea-7a66-4e1e-baef-ad8fee0668f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:17.291795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de7cea-7a66-4e1e-baef-ad8fee0668f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:17.292077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de7cea-7a66-4e1e-baef-ad8fee0668f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:17.292740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de7cea-7a66-4e1e-baef-ad8fee0668f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:17.295408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de7cea-7a66-4e1e-baef-ad8fee0668f6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:17.423294 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1178/2367] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:50:17 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:17.425150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de7cea-7a66-4e1e-baef-ad8fee0668f6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:17.466962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:17.485504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdcf538940a247b597c044d350dc6fe2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:17.487519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdcf538940a247b597c044d350dc6fe2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:17.567369 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de7cea-7a66-4e1e-baef-ad8fee0668f6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:17.574963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1de7da-fb65-40db-bac5-3c09dced7804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:17.575550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1de7da-fb65-40db-bac5-3c09dced7804 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:17.575829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1de7da-fb65-40db-bac5-3c09dced7804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:17.576105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1de7da-fb65-40db-bac5-3c09dced7804 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:17.576859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1de7da-fb65-40db-bac5-3c09dced7804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:17.581390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1de7da-fb65-40db-bac5-3c09dced7804 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:17.588240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdcf538940a247b597c044d350dc6fe2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:17.603614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdcf538940a247b597c044d350dc6fe2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:17.636420 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdcf538940a247b597c044d350dc6fe2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:17.654242 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1190/2368] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:50:17 2022] DELETE /identity/v3/users/bdcf538940a247b597c044d350dc6fe2 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:17.655424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:17.658930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:17.731071 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:17.780666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1179/2369] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:50:17 2022] POST /identity/v3/projects => generated 383 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:17.793603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1de7da-fb65-40db-bac5-3c09dced7804 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:17.804614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9a1de7da-fb65-40db-bac5-3c09dced7804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:17.838048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83280e4-6e96-4726-be65-1c5632fe6830 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:17.974430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1de7da-fb65-40db-bac5-3c09dced7804 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:17.986817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba0fd3-7f09-4c51-8698-aef161ec55ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:17.987434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0fd3-7f09-4c51-8698-aef161ec55ef admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:17.987692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0fd3-7f09-4c51-8698-aef161ec55ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:17.987968 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba0fd3-7f09-4c51-8698-aef161ec55ef admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:17.988624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba0fd3-7f09-4c51-8698-aef161ec55ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:17.992102 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba0fd3-7f09-4c51-8698-aef161ec55ef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:18.009479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d80c8-5a02-4a2c-946d-5911abf0022e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:18.020742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d80c8-5a02-4a2c-946d-5911abf0022e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:18.021295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d80c8-5a02-4a2c-946d-5911abf0022e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:18.021887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d80c8-5a02-4a2c-946d-5911abf0022e admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:18.023394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d80c8-5a02-4a2c-946d-5911abf0022e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:18.027860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d80c8-5a02-4a2c-946d-5911abf0022e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:18.130387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1180/2370] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:50:17 2022] POST /identity/v3/users => generated 292 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:18.130387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba0fd3-7f09-4c51-8698-aef161ec55ef admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:18.158772 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba0fd3-7f09-4c51-8698-aef161ec55ef admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:18.167160 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1191/2371] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:50:17 2022] POST /identity/v3/users => generated 287 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:18.168202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f46a9d-400c-4837-9fa9-0a2620b0d5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:18.168956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f46a9d-400c-4837-9fa9-0a2620b0d5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:18.169365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f46a9d-400c-4837-9fa9-0a2620b0d5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:18.169606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f46a9d-400c-4837-9fa9-0a2620b0d5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:18.170481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f46a9d-400c-4837-9fa9-0a2620b0d5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:18.171738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d80c8-5a02-4a2c-946d-5911abf0022e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:18.174003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f46a9d-400c-4837-9fa9-0a2620b0d5d9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:18.196988 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1181/2372] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:50:18.198544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d80c8-5a02-4a2c-946d-5911abf0022e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:18.203810 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f46a9d-400c-4837-9fa9-0a2620b0d5d9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:18.206303 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b7a2d-a78c-415e-a870-dedd0f9578ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:18.207216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b7a2d-a78c-415e-a870-dedd0f9578ee admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:18.208215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b7a2d-a78c-415e-a870-dedd0f9578ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:18.208709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b7a2d-a78c-415e-a870-dedd0f9578ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:18.209644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b7a2d-a78c-415e-a870-dedd0f9578ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:18.211617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b7a2d-a78c-415e-a870-dedd0f9578ee admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:18.225803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1192/2373] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:50:18.234130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b7a2d-a78c-415e-a870-dedd0f9578ee admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:18.252981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f46a9d-400c-4837-9fa9-0a2620b0d5d9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:18.257117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2f1dd-6d2d-43d8-80af-dcadc6f042fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:18.259971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2f1dd-6d2d-43d8-80af-dcadc6f042fc admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:18.259971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2f1dd-6d2d-43d8-80af-dcadc6f042fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:18.259971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2f1dd-6d2d-43d8-80af-dcadc6f042fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:18.259971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2f1dd-6d2d-43d8-80af-dcadc6f042fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:18.273448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2f1dd-6d2d-43d8-80af-dcadc6f042fc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:18.306827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1182/2374] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:50:18.311974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2f1dd-6d2d-43d8-80af-dcadc6f042fc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:18.343988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b7a2d-a78c-415e-a870-dedd0f9578ee admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:18.358793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f1610-579a-4aba-93dc-77b5718962fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:18.359039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2f1dd-6d2d-43d8-80af-dcadc6f042fc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:18.362887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f1610-579a-4aba-93dc-77b5718962fc admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:18.362887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f1610-579a-4aba-93dc-77b5718962fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:18.362887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f1610-579a-4aba-93dc-77b5718962fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:18.362887 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f1610-579a-4aba-93dc-77b5718962fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:18.365599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f1610-579a-4aba-93dc-77b5718962fc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:18.370542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a825588-be77-4c10-b39c-c0d8efc21c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:18.371367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a825588-be77-4c10-b39c-c0d8efc21c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:18.371696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a825588-be77-4c10-b39c-c0d8efc21c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:18.372042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a825588-be77-4c10-b39c-c0d8efc21c37 admin admin] PATH_INFO: `/v3/projects/2fb6bf0ad8fa459f8405ec50f42d9724/users/4ca81ee8daf14c3e82a09b03dbfa96fa/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:18.399529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1193/2375] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:50:18.404732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f1610-579a-4aba-93dc-77b5718962fc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:18.409314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a825588-be77-4c10-b39c-c0d8efc21c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb6bf0ad8fa459f8405ec50f42d9724, user_id=4ca81ee8daf14c3e82a09b03dbfa96fa, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:18.419162 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a825588-be77-4c10-b39c-c0d8efc21c37 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:18.419592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f1610-579a-4aba-93dc-77b5718962fc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:18.431302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230f8d8-b28d-4657-a853-66d303c9fb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:18.431763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230f8d8-b28d-4657-a853-66d303c9fb94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:18.432297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230f8d8-b28d-4657-a853-66d303c9fb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:18.432436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230f8d8-b28d-4657-a853-66d303c9fb94 admin admin] PATH_INFO: `/v3/projects/8035cc960d1f4aed981b1812e4abe6eb/users/708d05029cef49a4a7f7cbc78fce77ab/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:18.453208 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230f8d8-b28d-4657-a853-66d303c9fb94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8035cc960d1f4aed981b1812e4abe6eb, user_id=708d05029cef49a4a7f7cbc78fce77ab, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:18.463749 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230f8d8-b28d-4657-a853-66d303c9fb94 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:18.569057 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1183/2376] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:50:18 2022] PUT /identity/v3/projects/2fb6bf0ad8fa459f8405ec50f42d9724/users/4ca81ee8daf14c3e82a09b03dbfa96fa/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:18.569057 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a825588-be77-4c10-b39c-c0d8efc21c37 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:18.571930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4a825588-be77-4c10-b39c-c0d8efc21c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:18.598749 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1194/2377] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:50:18 2022] PUT /identity/v3/projects/8035cc960d1f4aed981b1812e4abe6eb/users/708d05029cef49a4a7f7cbc78fce77ab/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:50:18.600705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11baa0e0-917d-445e-b6df-9a7966ac73bd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:18.600988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11baa0e0-917d-445e-b6df-9a7966ac73bd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:18.601265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11baa0e0-917d-445e-b6df-9a7966ac73bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:18.692210 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-11baa0e0-917d-445e-b6df-9a7966ac73bd None None] MFA Rules not processed for user `4ca81ee8daf14c3e82a09b03dbfa96fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:18.696069 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11baa0e0-917d-445e-b6df-9a7966ac73bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:18.763073 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:18.816242 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a825588-be77-4c10-b39c-c0d8efc21c37 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:18.922217 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1195/2378] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:18.924104 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60334f-eabb-498a-926b-f7090b0e868f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:18.926501 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60334f-eabb-498a-926b-f7090b0e868f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:18.927580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60334f-eabb-498a-926b-f7090b0e868f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:18.942246 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:18.944929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:18.945969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:18.946734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:18.947241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] PATH_INFO: `/v3/projects/fc2c5ff315bf455eb365328b59c7531a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:18.955608 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc2c5ff315bf455eb365328b59c7531a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:18.959720 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:18.968073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fb60334f-eabb-498a-926b-f7090b0e868f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:18.969631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fb60334f-eabb-498a-926b-f7090b0e868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:19.006757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1196/2379] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:19.007570 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60334f-eabb-498a-926b-f7090b0e868f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:19.018417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fb60334f-eabb-498a-926b-f7090b0e868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:19.188565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc2c5ff315bf455eb365328b59c7531a', 'request_id': 'req-2b06ef7c-44b2-45a1-a2c5-0c149c117714'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:19.257990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60334f-eabb-498a-926b-f7090b0e868f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:19.279587 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc2c5ff315bf455eb365328b59c7531a', 'request_id': 'req-2b06ef7c-44b2-45a1-a2c5-0c149c117714'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:19.291296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc2c5ff315bf455eb365328b59c7531a', 'request_id': 'req-2b06ef7c-44b2-45a1-a2c5-0c149c117714'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:19.333783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] The token cache is being invalidate because project fc2c5ff315bf455eb365328b59c7531a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:50:19.334067 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:19.340518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:19.348558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1184/2380] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:50:18 2022] DELETE /identity/v3/projects/fc2c5ff315bf455eb365328b59c7531a => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:19.348558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:19.350733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:19.395322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b04a7a-86d2-48e6-b1c3-8c469ef2e67e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:19.396070 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b04a7a-86d2-48e6-b1c3-8c469ef2e67e service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:19.396422 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b04a7a-86d2-48e6-b1c3-8c469ef2e67e service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:19.397156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b04a7a-86d2-48e6-b1c3-8c469ef2e67e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:19.401531 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-77b04a7a-86d2-48e6-b1c3-8c469ef2e67e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:19.534640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06ef7c-44b2-45a1-a2c5-0c149c117714 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:19.542869 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cba651-df81-43de-a941-0608444e0c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:19.543557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cba651-df81-43de-a941-0608444e0c9e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:19.543866 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cba651-df81-43de-a941-0608444e0c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:19.544223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cba651-df81-43de-a941-0608444e0c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:19.550243 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-20cba651-df81-43de-a941-0608444e0c9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:19.665362 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:19.668120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b04a7a-86d2-48e6-b1c3-8c469ef2e67e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:19.671060 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b04a7a-86d2-48e6-b1c3-8c469ef2e67e service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:19.713136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1197/2381] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 705 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:19.717293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d74680-fd64-487b-8356-65463fa3c93f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:19.717593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d74680-fd64-487b-8356-65463fa3c93f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:19.717837 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d74680-fd64-487b-8356-65463fa3c93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:19.742357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-61d74680-fd64-487b-8356-65463fa3c93f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:19.743959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-61d74680-fd64-487b-8356-65463fa3c93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:19.762002 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cba651-df81-43de-a941-0608444e0c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:19.764896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cba651-df81-43de-a941-0608444e0c9e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:19.765364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1198/2382] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:19.774080 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4eb6c-3207-4178-b54c-cd4cbf6447d6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:19.774414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4eb6c-3207-4178-b54c-cd4cbf6447d6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:19.774705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4eb6c-3207-4178-b54c-cd4cbf6447d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:19.792750 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1185/2383] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:19.810841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-adc4eb6c-3207-4178-b54c-cd4cbf6447d6 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:19.812170 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-adc4eb6c-3207-4178-b54c-cd4cbf6447d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:19.826191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1199/2384] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:50:19.827397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cba651-df81-43de-a941-0608444e0c9e service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:19.849338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cba651-df81-43de-a941-0608444e0c9e service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:19.884436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11a1c4-7faa-493d-b0ed-d9efb2ad0e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:19.884775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a1c4-7faa-493d-b0ed-d9efb2ad0e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:19.885038 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a1c4-7faa-493d-b0ed-d9efb2ad0e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:19.885377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a1c4-7faa-493d-b0ed-d9efb2ad0e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:19.886191 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11a1c4-7faa-493d-b0ed-d9efb2ad0e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:50:19.886501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11a1c4-7faa-493d-b0ed-d9efb2ad0e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:19.888372 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11a1c4-7faa-493d-b0ed-d9efb2ad0e99 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:19.938962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1186/2385] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:50:19.944400 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4eb6c-3207-4178-b54c-cd4cbf6447d6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:19.955947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4eb6c-3207-4178-b54c-cd4cbf6447d6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:19.997863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869deea-d869-4111-81ce-59ef07c38360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:19.998809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869deea-d869-4111-81ce-59ef07c38360 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:20.000277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869deea-d869-4111-81ce-59ef07c38360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:20.003898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869deea-d869-4111-81ce-59ef07c38360 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:20.005135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869deea-d869-4111-81ce-59ef07c38360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:20.024362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869deea-d869-4111-81ce-59ef07c38360 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:20.186849 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1200/2386] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:50:19 2022] POST /identity/v3/projects => generated 385 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:50:20.191301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11a1c4-7faa-493d-b0ed-d9efb2ad0e99 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:20.204214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11a1c4-7faa-493d-b0ed-d9efb2ad0e99 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:20.314228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37b3c4-60f6-4d3c-be18-4c594f62481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:20.314832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37b3c4-60f6-4d3c-be18-4c594f62481f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:20.315178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37b3c4-60f6-4d3c-be18-4c594f62481f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:20.315527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37b3c4-60f6-4d3c-be18-4c594f62481f admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:20.316240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37b3c4-60f6-4d3c-be18-4c594f62481f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:20.318424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37b3c4-60f6-4d3c-be18-4c594f62481f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:20.570964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1187/2387] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:50:20 2022] POST /identity/v3/users => generated 293 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:50:20.585797 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869deea-d869-4111-81ce-59ef07c38360 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:20.608239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869deea-d869-4111-81ce-59ef07c38360 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:20.618188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bfc79-3898-4c50-9129-09361eface1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:20.619379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bfc79-3898-4c50-9129-09361eface1f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:20.619926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bfc79-3898-4c50-9129-09361eface1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:20.620800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bfc79-3898-4c50-9129-09361eface1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:20.622302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bfc79-3898-4c50-9129-09361eface1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:20.630101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bfc79-3898-4c50-9129-09361eface1f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:20.657102 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37b3c4-60f6-4d3c-be18-4c594f62481f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:20.659293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1201/2388] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:50:20.669335 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37b3c4-60f6-4d3c-be18-4c594f62481f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:20.675190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07830078-5a0b-40e6-8028-517c7610e2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:20.675884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07830078-5a0b-40e6-8028-517c7610e2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:20.676173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07830078-5a0b-40e6-8028-517c7610e2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:20.676447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07830078-5a0b-40e6-8028-517c7610e2b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:20.677136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07830078-5a0b-40e6-8028-517c7610e2b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:20.678905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07830078-5a0b-40e6-8028-517c7610e2b9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:20.692920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1188/2389] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:50:20.702923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bfc79-3898-4c50-9129-09361eface1f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:20.732481 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bfc79-3898-4c50-9129-09361eface1f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:20.745984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ef9ab-d6c5-4170-8a1f-bf92374e842b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:20.747361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ef9ab-d6c5-4170-8a1f-bf92374e842b admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:20.747910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ef9ab-d6c5-4170-8a1f-bf92374e842b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:20.748434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ef9ab-d6c5-4170-8a1f-bf92374e842b admin admin] PATH_INFO: `/v3/projects/9df149cbc6ff4b94bbcee2ac91dd5ecd/users/be788861c04146c6bd7558bab26542a1/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:20.785274 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ef9ab-d6c5-4170-8a1f-bf92374e842b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df149cbc6ff4b94bbcee2ac91dd5ecd, user_id=be788861c04146c6bd7558bab26542a1, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:20.791671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ef9ab-d6c5-4170-8a1f-bf92374e842b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:20.897841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:20.913324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1202/2390] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:50:20 2022] PUT /identity/v3/projects/9df149cbc6ff4b94bbcee2ac91dd5ecd/users/be788861c04146c6bd7558bab26542a1/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:21.000731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b9561-6976-494f-87ff-c5ab9751cf33 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:21.001075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b9561-6976-494f-87ff-c5ab9751cf33 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:21.001310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b9561-6976-494f-87ff-c5ab9751cf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:21.059281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-280b9561-6976-494f-87ff-c5ab9751cf33 None None] MFA Rules not processed for user `be788861c04146c6bd7558bab26542a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:21.060989 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-280b9561-6976-494f-87ff-c5ab9751cf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:21.290187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1189/2391] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:21.294526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ef9ab-d6c5-4170-8a1f-bf92374e842b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:21.450055 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ef9ab-d6c5-4170-8a1f-bf92374e842b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:21.459527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4423646-9f93-42f9-b71f-c0f11d4d1ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:21.460152 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4423646-9f93-42f9-b71f-c0f11d4d1ded admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:21.460426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4423646-9f93-42f9-b71f-c0f11d4d1ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:21.460695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4423646-9f93-42f9-b71f-c0f11d4d1ded admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:21.461353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4423646-9f93-42f9-b71f-c0f11d4d1ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:21.463958 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4423646-9f93-42f9-b71f-c0f11d4d1ded admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:21.623829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1203/2392] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:50:21 2022] POST /identity/v3/projects => generated 383 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:21.625549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b9561-6976-494f-87ff-c5ab9751cf33 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:21.653859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b9561-6976-494f-87ff-c5ab9751cf33 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:21.750504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:21.929397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205266a-2955-493e-9194-3fbf5ef55aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:21.930401 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205266a-2955-493e-9194-3fbf5ef55aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:21.930868 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205266a-2955-493e-9194-3fbf5ef55aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:21.931312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205266a-2955-493e-9194-3fbf5ef55aa1 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:21.932393 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205266a-2955-493e-9194-3fbf5ef55aa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:21.936238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205266a-2955-493e-9194-3fbf5ef55aa1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:22.067637 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1190/2393] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:50:21 2022] POST /identity/v3/users => generated 292 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:22.072687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4423646-9f93-42f9-b71f-c0f11d4d1ded admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:22.084882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4423646-9f93-42f9-b71f-c0f11d4d1ded admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:22.089969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da911da-f72d-45c0-98cb-9c1b154d145e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:22.093840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da911da-f72d-45c0-98cb-9c1b154d145e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:22.093840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da911da-f72d-45c0-98cb-9c1b154d145e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:22.093840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da911da-f72d-45c0-98cb-9c1b154d145e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:22.093840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da911da-f72d-45c0-98cb-9c1b154d145e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:22.093840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da911da-f72d-45c0-98cb-9c1b154d145e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:22.109692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1204/2394] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:50:22.111187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205266a-2955-493e-9194-3fbf5ef55aa1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:22.127416 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205266a-2955-493e-9194-3fbf5ef55aa1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:22.133502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd38cdc-01ee-4e71-b97f-a485febb3b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:22.134425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd38cdc-01ee-4e71-b97f-a485febb3b17 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:22.135003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd38cdc-01ee-4e71-b97f-a485febb3b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:22.135400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd38cdc-01ee-4e71-b97f-a485febb3b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:22.136174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd38cdc-01ee-4e71-b97f-a485febb3b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:22.137910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd38cdc-01ee-4e71-b97f-a485febb3b17 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:22.157107 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1191/2395] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:50:22.161402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da911da-f72d-45c0-98cb-9c1b154d145e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:22.174352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da911da-f72d-45c0-98cb-9c1b154d145e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:22.184468 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217cffb-aaef-46fb-9b5e-2a7585f6236b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:22.185150 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217cffb-aaef-46fb-9b5e-2a7585f6236b admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:22.187173 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217cffb-aaef-46fb-9b5e-2a7585f6236b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:22.187464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217cffb-aaef-46fb-9b5e-2a7585f6236b admin admin] PATH_INFO: `/v3/projects/cdcff85be63649278c12d4ad16ac0384/users/a1b339b96ca4416ab12bd0eb75b48f4b/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:22.238692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217cffb-aaef-46fb-9b5e-2a7585f6236b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdcff85be63649278c12d4ad16ac0384, user_id=a1b339b96ca4416ab12bd0eb75b48f4b, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:22.241435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217cffb-aaef-46fb-9b5e-2a7585f6236b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:22.351586 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1205/2396] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:50:22 2022] PUT /identity/v3/projects/cdcff85be63649278c12d4ad16ac0384/users/a1b339b96ca4416ab12bd0eb75b48f4b/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:50:22.367357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a92123-d9cd-43bc-9c98-a8a1ff4aad57 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:22.367357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a92123-d9cd-43bc-9c98-a8a1ff4aad57 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:22.367357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a92123-d9cd-43bc-9c98-a8a1ff4aad57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:22.443216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f6a92123-d9cd-43bc-9c98-a8a1ff4aad57 None None] MFA Rules not processed for user `a1b339b96ca4416ab12bd0eb75b48f4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:22.448039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6a92123-d9cd-43bc-9c98-a8a1ff4aad57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:22.590218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1192/2397] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:22.598006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217cffb-aaef-46fb-9b5e-2a7585f6236b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:22.752393 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217cffb-aaef-46fb-9b5e-2a7585f6236b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:22.758188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a62df-7278-4a7d-a3a7-154326ab33aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:22.758828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a62df-7278-4a7d-a3a7-154326ab33aa admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:22.759104 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a62df-7278-4a7d-a3a7-154326ab33aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:22.759408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a62df-7278-4a7d-a3a7-154326ab33aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:22.760050 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a62df-7278-4a7d-a3a7-154326ab33aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:22.762109 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a62df-7278-4a7d-a3a7-154326ab33aa admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:22.867085 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1206/2398] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:50:22 2022] POST /identity/v3/projects => generated 385 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:22.868368 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a92123-d9cd-43bc-9c98-a8a1ff4aad57 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:22.893219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a92123-d9cd-43bc-9c98-a8a1ff4aad57 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:23.036901 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f1438-9bab-4926-870f-482a6c3b3821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:23.037783 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f1438-9bab-4926-870f-482a6c3b3821 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:23.038264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f1438-9bab-4926-870f-482a6c3b3821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:23.038705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f1438-9bab-4926-870f-482a6c3b3821 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:23.039525 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f1438-9bab-4926-870f-482a6c3b3821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:23.042458 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f1438-9bab-4926-870f-482a6c3b3821 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:23.153524 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1193/2399] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:50:22 2022] POST /identity/v3/users => generated 299 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:50:23.158554 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a62df-7278-4a7d-a3a7-154326ab33aa admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:23.172388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a62df-7278-4a7d-a3a7-154326ab33aa admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:23.182364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6c32d-77b7-4e02-b4a1-2a2150198552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:23.183129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6c32d-77b7-4e02-b4a1-2a2150198552 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:23.183413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6c32d-77b7-4e02-b4a1-2a2150198552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:23.183848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6c32d-77b7-4e02-b4a1-2a2150198552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:23.184971 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6c32d-77b7-4e02-b4a1-2a2150198552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:23.186713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6c32d-77b7-4e02-b4a1-2a2150198552 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:23.209104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f1438-9bab-4926-870f-482a6c3b3821 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:23.222207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1207/2400] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:50:23.239923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f1438-9bab-4926-870f-482a6c3b3821 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:23.258329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22808ea3-8164-4b19-94dd-2ce95a489899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:23.258329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22808ea3-8164-4b19-94dd-2ce95a489899 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:23.258329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22808ea3-8164-4b19-94dd-2ce95a489899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:23.258329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22808ea3-8164-4b19-94dd-2ce95a489899 admin admin] PATH_INFO: `/v3/projects/d24c465898124238b0092d58a9f9aff7/users/c71ea606bed14e86956157d6bfa2ed10/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:23.295860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22808ea3-8164-4b19-94dd-2ce95a489899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d24c465898124238b0092d58a9f9aff7, user_id=c71ea606bed14e86956157d6bfa2ed10, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:23.295860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22808ea3-8164-4b19-94dd-2ce95a489899 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:23.431922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:23.437826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1194/2401] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:50:23 2022] PUT /identity/v3/projects/d24c465898124238b0092d58a9f9aff7/users/c71ea606bed14e86956157d6bfa2ed10/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:23.444687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6c32d-77b7-4e02-b4a1-2a2150198552 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:23.466773 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6c32d-77b7-4e02-b4a1-2a2150198552 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:23.507327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:23.611544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f8406-d704-469e-9069-b1592eb5751d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:23.612238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f8406-d704-469e-9069-b1592eb5751d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:23.612523 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f8406-d704-469e-9069-b1592eb5751d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:23.612793 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f8406-d704-469e-9069-b1592eb5751d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:23.613482 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f8406-d704-469e-9069-b1592eb5751d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:23.615452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f8406-d704-469e-9069-b1592eb5751d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:23.650395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1208/2402] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:50:23.651036 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22808ea3-8164-4b19-94dd-2ce95a489899 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:23.672424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22808ea3-8164-4b19-94dd-2ce95a489899 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:23.681593 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46babc19-0607-4ca8-aecd-056b9873a0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:23.682304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46babc19-0607-4ca8-aecd-056b9873a0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:23.682586 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46babc19-0607-4ca8-aecd-056b9873a0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:23.682982 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46babc19-0607-4ca8-aecd-056b9873a0f9 admin admin] PATH_INFO: `/v3/projects/d24c465898124238b0092d58a9f9aff7/users/c71ea606bed14e86956157d6bfa2ed10/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:23.685064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46babc19-0607-4ca8-aecd-056b9873a0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d24c465898124238b0092d58a9f9aff7, user_id=c71ea606bed14e86956157d6bfa2ed10, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:23.687899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46babc19-0607-4ca8-aecd-056b9873a0f9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:23.733918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1195/2403] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:50:23 2022] PUT /identity/v3/projects/d24c465898124238b0092d58a9f9aff7/users/c71ea606bed14e86956157d6bfa2ed10/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:23.741989 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec5921-8222-4157-83f6-994fc23bfc9a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:23.742537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec5921-8222-4157-83f6-994fc23bfc9a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:23.743023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec5921-8222-4157-83f6-994fc23bfc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:23.775395 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46babc19-0607-4ca8-aecd-056b9873a0f9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:23.779673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-46babc19-0607-4ca8-aecd-056b9873a0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:23.847888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-deec5921-8222-4157-83f6-994fc23bfc9a None None] MFA Rules not processed for user `c71ea606bed14e86956157d6bfa2ed10`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:23.849147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-deec5921-8222-4157-83f6-994fc23bfc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:23.975560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46babc19-0607-4ca8-aecd-056b9873a0f9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:23.982724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:23.983756 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:23.984036 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:23.984308 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] PATH_INFO: `/v3/users/4ca81ee8daf14c3e82a09b03dbfa96fa` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:23.985330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1209/2404] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:23.985753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ca81ee8daf14c3e82a09b03dbfa96fa)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:23.988082 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:23.998082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec5921-8222-4157-83f6-994fc23bfc9a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:23.999590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-deec5921-8222-4157-83f6-994fc23bfc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:24.149507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec5921-8222-4157-83f6-994fc23bfc9a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:24.154712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f5856-3bc1-4531-ab11-3d528a83607e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:24.155426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f5856-3bc1-4531-ab11-3d528a83607e service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:24.155694 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f5856-3bc1-4531-ab11-3d528a83607e service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:24.155963 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f5856-3bc1-4531-ab11-3d528a83607e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:24.373978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f5856-3bc1-4531-ab11-3d528a83607e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:24.375498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f5856-3bc1-4531-ab11-3d528a83607e service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:24.409360 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1210/2405] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:24.499308 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:24.578935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ca81ee8daf14c3e82a09b03dbfa96fa'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:24.635816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ca81ee8daf14c3e82a09b03dbfa96fa'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:24.748209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ca81ee8daf14c3e82a09b03dbfa96fa'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:24.768978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ca81ee8daf14c3e82a09b03dbfa96fa'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:24.768978 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ca81ee8daf14c3e82a09b03dbfa96fa'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:24.791953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1196/2406] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:50:23 2022] DELETE /identity/v3/users/4ca81ee8daf14c3e82a09b03dbfa96fa => generated 0 bytes in 1017 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:50:24.796313 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1bf1a-d190-4171-b229-215b2c38fe8b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:24.796585 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1bf1a-d190-4171-b229-215b2c38fe8b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:24.796852 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1bf1a-d190-4171-b229-215b2c38fe8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:24.858772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1ed1bf1a-d190-4171-b229-215b2c38fe8b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:24.861192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1ed1bf1a-d190-4171-b229-215b2c38fe8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:25.095924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1211/2407] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:25.128766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:25.131348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:25.310378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16208d28-2584-42f2-8f03-17cd66f1762f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:25.327118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1efd47-6156-421b-ab94-75889f33548b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:25.327850 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1efd47-6156-421b-ab94-75889f33548b service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:25.332166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1efd47-6156-421b-ab94-75889f33548b service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:25.332166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1efd47-6156-421b-ab94-75889f33548b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:25.341606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1efd47-6156-421b-ab94-75889f33548b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:25.344113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1efd47-6156-421b-ab94-75889f33548b service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:25.360370 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1197/2408] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:27.044485 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1bf1a-d190-4171-b229-215b2c38fe8b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:27.047748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1ed1bf1a-d190-4171-b229-215b2c38fe8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:27.068141 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1bf1a-d190-4171-b229-215b2c38fe8b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:27.073559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8037b2-796b-425e-9b44-6e623f0f9ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:27.074400 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8037b2-796b-425e-9b44-6e623f0f9ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:27.074875 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8037b2-796b-425e-9b44-6e623f0f9ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:27.075037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8037b2-796b-425e-9b44-6e623f0f9ea3 admin admin] PATH_INFO: `/v3/projects/2fb6bf0ad8fa459f8405ec50f42d9724` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:27.076682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8037b2-796b-425e-9b44-6e623f0f9ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb6bf0ad8fa459f8405ec50f42d9724)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:27.078612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8037b2-796b-425e-9b44-6e623f0f9ea3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:27.242273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c8037b2-796b-425e-9b44-6e623f0f9ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb6bf0ad8fa459f8405ec50f42d9724', 'request_id': 'req-7c8037b2-796b-425e-9b44-6e623f0f9ea3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:27.323458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c8037b2-796b-425e-9b44-6e623f0f9ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb6bf0ad8fa459f8405ec50f42d9724', 'request_id': 'req-7c8037b2-796b-425e-9b44-6e623f0f9ea3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:27.334709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c8037b2-796b-425e-9b44-6e623f0f9ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb6bf0ad8fa459f8405ec50f42d9724', 'request_id': 'req-7c8037b2-796b-425e-9b44-6e623f0f9ea3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:27.368870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c8037b2-796b-425e-9b44-6e623f0f9ea3 admin admin] The token cache is being invalidate because project 2fb6bf0ad8fa459f8405ec50f42d9724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:50:27.369571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c8037b2-796b-425e-9b44-6e623f0f9ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:27.370955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7c8037b2-796b-425e-9b44-6e623f0f9ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:27.374909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1212/2409] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:50:27 2022] DELETE /identity/v3/projects/2fb6bf0ad8fa459f8405ec50f42d9724 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:27.379589 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ff755-b140-4762-884a-0c40b41e3d74 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:27.380501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ff755-b140-4762-884a-0c40b41e3d74 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:27.380501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ff755-b140-4762-884a-0c40b41e3d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:27.406684 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4b9ff755-b140-4762-884a-0c40b41e3d74 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:27.408497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4b9ff755-b140-4762-884a-0c40b41e3d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:27.416790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:27.542118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1198/2410] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:27.550764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeec35-6e0e-4480-9735-3da4ae265e6a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:27.556917 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeec35-6e0e-4480-9735-3da4ae265e6a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:27.556917 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aeec35-6e0e-4480-9735-3da4ae265e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:27.593155 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-40aeec35-6e0e-4480-9735-3da4ae265e6a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:27.598550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-40aeec35-6e0e-4480-9735-3da4ae265e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:27.633040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1213/2411] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:27.637568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ff755-b140-4762-884a-0c40b41e3d74 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:27.662370 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ff755-b140-4762-884a-0c40b41e3d74 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:27.665064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480e008-a18c-4f68-8b44-5839a78b8ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:27.665696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480e008-a18c-4f68-8b44-5839a78b8ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:27.665977 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480e008-a18c-4f68-8b44-5839a78b8ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:27.666368 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480e008-a18c-4f68-8b44-5839a78b8ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:27.667170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480e008-a18c-4f68-8b44-5839a78b8ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:50:27.667482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480e008-a18c-4f68-8b44-5839a78b8ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:27.669246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480e008-a18c-4f68-8b44-5839a78b8ca7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:27.683936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1199/2412] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:50:27.689548 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeec35-6e0e-4480-9735-3da4ae265e6a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:27.708271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aeec35-6e0e-4480-9735-3da4ae265e6a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:27.708805 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4b130-47cd-476f-841c-9eff5a4ea892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:27.709478 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4b130-47cd-476f-841c-9eff5a4ea892 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:27.709764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4b130-47cd-476f-841c-9eff5a4ea892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:27.710034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4b130-47cd-476f-841c-9eff5a4ea892 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:27.710724 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4b130-47cd-476f-841c-9eff5a4ea892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:27.712796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4b130-47cd-476f-841c-9eff5a4ea892 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:27.864402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1214/2413] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:50:27 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:27.868162 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480e008-a18c-4f68-8b44-5839a78b8ca7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:27.879382 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480e008-a18c-4f68-8b44-5839a78b8ca7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:27.962909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39500e-7dc2-402e-835e-c32aff0861bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:27.963491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39500e-7dc2-402e-835e-c32aff0861bb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:27.963897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39500e-7dc2-402e-835e-c32aff0861bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:27.964812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39500e-7dc2-402e-835e-c32aff0861bb admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:27.965015 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39500e-7dc2-402e-835e-c32aff0861bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:27.971364 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39500e-7dc2-402e-835e-c32aff0861bb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:28.146395 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1200/2414] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:50:27 2022] POST /identity/v3/users => generated 287 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:28.151967 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4b130-47cd-476f-841c-9eff5a4ea892 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:28.163942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4b130-47cd-476f-841c-9eff5a4ea892 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:28.170009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeef71e-54c3-40c6-a6de-1dd25ed87e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:28.170783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeef71e-54c3-40c6-a6de-1dd25ed87e62 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:28.171158 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeef71e-54c3-40c6-a6de-1dd25ed87e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:28.171529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeef71e-54c3-40c6-a6de-1dd25ed87e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:28.172284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeef71e-54c3-40c6-a6de-1dd25ed87e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:28.174821 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeef71e-54c3-40c6-a6de-1dd25ed87e62 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:28.185823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1215/2415] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:50:28.202692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39500e-7dc2-402e-835e-c32aff0861bb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:28.214833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39500e-7dc2-402e-835e-c32aff0861bb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:28.220849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430d1e0-7228-440f-80e4-89abc26fe607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:28.221653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430d1e0-7228-440f-80e4-89abc26fe607 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:28.221935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430d1e0-7228-440f-80e4-89abc26fe607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:28.222237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430d1e0-7228-440f-80e4-89abc26fe607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:28.223003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430d1e0-7228-440f-80e4-89abc26fe607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:28.224757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430d1e0-7228-440f-80e4-89abc26fe607 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:28.238662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1201/2416] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:50:28.242874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeef71e-54c3-40c6-a6de-1dd25ed87e62 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:28.256174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeef71e-54c3-40c6-a6de-1dd25ed87e62 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:28.278736 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31ae44-88f7-4518-8dbe-91047b4ce2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:28.279409 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31ae44-88f7-4518-8dbe-91047b4ce2ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:28.279682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31ae44-88f7-4518-8dbe-91047b4ce2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:28.279946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31ae44-88f7-4518-8dbe-91047b4ce2ca admin admin] PATH_INFO: `/v3/projects/dab14bff5cee4e8fad0381a94aa9388f/users/c6d30261934a429dbcc38ac0f963855d/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:28.300963 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31ae44-88f7-4518-8dbe-91047b4ce2ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=dab14bff5cee4e8fad0381a94aa9388f, user_id=c6d30261934a429dbcc38ac0f963855d, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:28.306921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31ae44-88f7-4518-8dbe-91047b4ce2ca admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:28.342070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430d1e0-7228-440f-80e4-89abc26fe607 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:28.356777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e430d1e0-7228-440f-80e4-89abc26fe607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:28.434309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430d1e0-7228-440f-80e4-89abc26fe607 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:28.434937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1216/2417] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:50:28 2022] PUT /identity/v3/projects/dab14bff5cee4e8fad0381a94aa9388f/users/c6d30261934a429dbcc38ac0f963855d/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:28.443093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7141f-8e2c-40f8-9a8f-b3d1efc2db0a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:28.443380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7141f-8e2c-40f8-9a8f-b3d1efc2db0a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:28.443662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7141f-8e2c-40f8-9a8f-b3d1efc2db0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:28.477785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f1f7141f-8e2c-40f8-9a8f-b3d1efc2db0a None None] MFA Rules not processed for user `c6d30261934a429dbcc38ac0f963855d`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:28.479877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f1f7141f-8e2c-40f8-9a8f-b3d1efc2db0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:28.486939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3ad65-fd35-4ce9-b692-91eb6b37b2e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:28.487552 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3ad65-fd35-4ce9-b692-91eb6b37b2e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:28.487827 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3ad65-fd35-4ce9-b692-91eb6b37b2e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:28.488095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3ad65-fd35-4ce9-b692-91eb6b37b2e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:28.488717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:28.489910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-97f3ad65-fd35-4ce9-b692-91eb6b37b2e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:28.575493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3ad65-fd35-4ce9-b692-91eb6b37b2e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:28.577578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3ad65-fd35-4ce9-b692-91eb6b37b2e6 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:28.581859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1217/2418] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:50:28.589560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7141f-8e2c-40f8-9a8f-b3d1efc2db0a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:28.591910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f1f7141f-8e2c-40f8-9a8f-b3d1efc2db0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:28.593136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1202/2419] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:50:28.608473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7141f-8e2c-40f8-9a8f-b3d1efc2db0a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:28.614428 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8fbb5e-0eac-492a-a57d-66dad625f5b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:28.615299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8fbb5e-0eac-492a-a57d-66dad625f5b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:28.616759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8fbb5e-0eac-492a-a57d-66dad625f5b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:28.616759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8fbb5e-0eac-492a-a57d-66dad625f5b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:28.626535 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8fbb5e-0eac-492a-a57d-66dad625f5b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:28.628450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8fbb5e-0eac-492a-a57d-66dad625f5b2 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:28.651272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1218/2420] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:29.723030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:30.286380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcabac-2eeb-468d-80e3-5b5283d31a23 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:30.286763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcabac-2eeb-468d-80e3-5b5283d31a23 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:30.287135 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcabac-2eeb-468d-80e3-5b5283d31a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:30.359691 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-90bcabac-2eeb-468d-80e3-5b5283d31a23 None None] MFA Rules not processed for user `708d05029cef49a4a7f7cbc78fce77ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:30.361951 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-90bcabac-2eeb-468d-80e3-5b5283d31a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:30.487238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1203/2421] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:30.491963 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8fbb5e-0eac-492a-a57d-66dad625f5b2 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:30.493667 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9c8fbb5e-0eac-492a-a57d-66dad625f5b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:30.630695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8fbb5e-0eac-492a-a57d-66dad625f5b2 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:30.636282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc5767-5093-4511-9096-1f363f553cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:30.637269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc5767-5093-4511-9096-1f363f553cdc admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:30.637758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc5767-5093-4511-9096-1f363f553cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:30.638299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc5767-5093-4511-9096-1f363f553cdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:30.639299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc5767-5093-4511-9096-1f363f553cdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:30.641658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc5767-5093-4511-9096-1f363f553cdc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:30.783432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1219/2422] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:50:30 2022] POST /identity/v3/projects => generated 373 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:30.786816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcabac-2eeb-468d-80e3-5b5283d31a23 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:30.807648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcabac-2eeb-468d-80e3-5b5283d31a23 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:30.900661 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc5767-5093-4511-9096-1f363f553cdc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:30.909434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a8d9e-caed-4b03-90d5-27bfaab91c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:30.910049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a8d9e-caed-4b03-90d5-27bfaab91c83 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:30.910515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a8d9e-caed-4b03-90d5-27bfaab91c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:30.910787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a8d9e-caed-4b03-90d5-27bfaab91c83 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:30.911926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a8d9e-caed-4b03-90d5-27bfaab91c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:30.914311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a8d9e-caed-4b03-90d5-27bfaab91c83 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:30.979726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc5767-5093-4511-9096-1f363f553cdc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:30.994213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d25af6-926c-4a75-8839-a5fe68981c5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:30.995137 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d25af6-926c-4a75-8839-a5fe68981c5c service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:30.996741 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d25af6-926c-4a75-8839-a5fe68981c5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:30.997188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d25af6-926c-4a75-8839-a5fe68981c5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:31.003187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-54d25af6-926c-4a75-8839-a5fe68981c5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:31.066052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1204/2423] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:50:30 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:31.071495 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a8d9e-caed-4b03-90d5-27bfaab91c83 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:31.084982 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a8d9e-caed-4b03-90d5-27bfaab91c83 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:31.091739 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4502e-54b8-4a04-9b99-7c8a4febd029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:31.092687 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4502e-54b8-4a04-9b99-7c8a4febd029 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:31.093321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4502e-54b8-4a04-9b99-7c8a4febd029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:31.093870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4502e-54b8-4a04-9b99-7c8a4febd029 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:31.095306 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4502e-54b8-4a04-9b99-7c8a4febd029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:31.097419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4502e-54b8-4a04-9b99-7c8a4febd029 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:31.107230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:31.128288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1205/2424] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:50:31.130204 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4502e-54b8-4a04-9b99-7c8a4febd029 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:31.147373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4502e-54b8-4a04-9b99-7c8a4febd029 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:31.160044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d25af6-926c-4a75-8839-a5fe68981c5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:31.160729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcae61f-5905-469b-8d70-ef60ef38929b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:31.161902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcae61f-5905-469b-8d70-ef60ef38929b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:31.162358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcae61f-5905-469b-8d70-ef60ef38929b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:31.162470 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d25af6-926c-4a75-8839-a5fe68981c5c service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:31.162634 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcae61f-5905-469b-8d70-ef60ef38929b admin admin] PATH_INFO: `/v3/projects/5cbdbd56714e4f1b82723835cb318fd3/users/db76d309360e4403885b1a64701ae527/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:31.176809 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1220/2425] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:50:31.193973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcae61f-5905-469b-8d70-ef60ef38929b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbdbd56714e4f1b82723835cb318fd3, user_id=db76d309360e4403885b1a64701ae527, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:31.201624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcae61f-5905-469b-8d70-ef60ef38929b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:31.317333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1206/2426] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:50:31 2022] PUT /identity/v3/projects/5cbdbd56714e4f1b82723835cb318fd3/users/db76d309360e4403885b1a64701ae527/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:31.317450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d25af6-926c-4a75-8839-a5fe68981c5c service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:31.327826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d25af6-926c-4a75-8839-a5fe68981c5c service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:31.400559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6f6e2-332e-4d92-a371-326276b5b17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:31.401328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6f6e2-332e-4d92-a371-326276b5b17b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:31.401696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6f6e2-332e-4d92-a371-326276b5b17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:31.403215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6f6e2-332e-4d92-a371-326276b5b17b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:31.403215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6f6e2-332e-4d92-a371-326276b5b17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:31.405587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6f6e2-332e-4d92-a371-326276b5b17b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:31.439659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1221/2427] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:50:31.443583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcae61f-5905-469b-8d70-ef60ef38929b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:31.455326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcae61f-5905-469b-8d70-ef60ef38929b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:31.463581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edd559-e443-4b5b-9f0d-03f1bc0711ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:31.466798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edd559-e443-4b5b-9f0d-03f1bc0711ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:31.466798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edd559-e443-4b5b-9f0d-03f1bc0711ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:31.466798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edd559-e443-4b5b-9f0d-03f1bc0711ac admin admin] PATH_INFO: `/v3/projects/5cbdbd56714e4f1b82723835cb318fd3/users/db76d309360e4403885b1a64701ae527/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:31.467116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85edd559-e443-4b5b-9f0d-03f1bc0711ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbdbd56714e4f1b82723835cb318fd3, user_id=db76d309360e4403885b1a64701ae527, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:31.469731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85edd559-e443-4b5b-9f0d-03f1bc0711ac admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:31.482697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1207/2428] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:50:31 2022] PUT /identity/v3/projects/5cbdbd56714e4f1b82723835cb318fd3/users/db76d309360e4403885b1a64701ae527/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:31.658227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:38.896740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab6b48-5878-4eeb-b9c1-344ff84d05ef None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:38.897268 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab6b48-5878-4eeb-b9c1-344ff84d05ef None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:38.897735 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab6b48-5878-4eeb-b9c1-344ff84d05ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:38.936888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-71ab6b48-5878-4eeb-b9c1-344ff84d05ef None None] MFA Rules not processed for user `db76d309360e4403885b1a64701ae527`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:38.939843 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-71ab6b48-5878-4eeb-b9c1-344ff84d05ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:39.041178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1222/2429] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:39.055162 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edd559-e443-4b5b-9f0d-03f1bc0711ac admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:39.056968 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-85edd559-e443-4b5b-9f0d-03f1bc0711ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:39.169833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edd559-e443-4b5b-9f0d-03f1bc0711ac admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:39.187345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d7434-a629-4e00-8262-7fc2ad19cdae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:39.188090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d7434-a629-4e00-8262-7fc2ad19cdae service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:39.189029 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d7434-a629-4e00-8262-7fc2ad19cdae service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:39.189421 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d7434-a629-4e00-8262-7fc2ad19cdae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:39.270981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d7434-a629-4e00-8262-7fc2ad19cdae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:39.272682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d7434-a629-4e00-8262-7fc2ad19cdae service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:39.287985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1208/2430] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:40.889278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab6b48-5878-4eeb-b9c1-344ff84d05ef None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:40.892136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-71ab6b48-5878-4eeb-b9c1-344ff84d05ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:40.970074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:40.986025 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab6b48-5878-4eeb-b9c1-344ff84d05ef None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:40.990994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:40.991691 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:40.992012 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:40.992350 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] PATH_INFO: `/v3/users/be788861c04146c6bd7558bab26542a1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:40.993573 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be788861c04146c6bd7558bab26542a1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:41.002362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:41.236444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be788861c04146c6bd7558bab26542a1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:41.237303 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be788861c04146c6bd7558bab26542a1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:41.325023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be788861c04146c6bd7558bab26542a1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:41.332994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be788861c04146c6bd7558bab26542a1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:41.347787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be788861c04146c6bd7558bab26542a1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:41.358983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1223/2431] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:50:40 2022] DELETE /identity/v3/users/be788861c04146c6bd7558bab26542a1 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:41.363673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e7e12-dc37-4074-a2a8-acb10d1a836a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:41.364175 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e7e12-dc37-4074-a2a8-acb10d1a836a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:41.364430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e7e12-dc37-4074-a2a8-acb10d1a836a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:41.385375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f67e7e12-dc37-4074-a2a8-acb10d1a836a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:41.386782 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f67e7e12-dc37-4074-a2a8-acb10d1a836a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:41.450606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:41.460975 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1209/2432] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:41.472099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:41.473970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:41.499105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22795dde-2e1b-415f-8326-05dd79cd9b49 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:41.499511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22795dde-2e1b-415f-8326-05dd79cd9b49 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:41.499674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22795dde-2e1b-415f-8326-05dd79cd9b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:41.523041 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-22795dde-2e1b-415f-8326-05dd79cd9b49 None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:41.529066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-22795dde-2e1b-415f-8326-05dd79cd9b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:41.545395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed3ad3-eeb9-4c11-946f-564292b6b890 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:41.550712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b937d8-d0bb-4355-b110-cd4ca57bdf6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:41.551437 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b937d8-d0bb-4355-b110-cd4ca57bdf6a service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:41.551968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b937d8-d0bb-4355-b110-cd4ca57bdf6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:41.552423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b937d8-d0bb-4355-b110-cd4ca57bdf6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:41.565429 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b937d8-d0bb-4355-b110-cd4ca57bdf6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:41.567180 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b937d8-d0bb-4355-b110-cd4ca57bdf6a service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:41.586869 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1224/2433] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:41.597803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1210/2434] 10.222.0.44 () {58 vars in 967 bytes} [Sun Sep 11 08:50:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:41.623298 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b937d8-d0bb-4355-b110-cd4ca57bdf6a service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:41.624897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b8b937d8-d0bb-4355-b110-cd4ca57bdf6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:41.710008 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b937d8-d0bb-4355-b110-cd4ca57bdf6a service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:41.720343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f90f41-2a68-4bda-b6e1-6010ad94ea7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:41.720976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f90f41-2a68-4bda-b6e1-6010ad94ea7f service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:41.721311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f90f41-2a68-4bda-b6e1-6010ad94ea7f service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:41.721644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f90f41-2a68-4bda-b6e1-6010ad94ea7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:41.731394 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f90f41-2a68-4bda-b6e1-6010ad94ea7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:41.733014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f90f41-2a68-4bda-b6e1-6010ad94ea7f service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:41.742568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1225/2435] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:50:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:50:42.403672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22795dde-2e1b-415f-8326-05dd79cd9b49 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:42.405614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-22795dde-2e1b-415f-8326-05dd79cd9b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:42.434582 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22795dde-2e1b-415f-8326-05dd79cd9b49 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:42.443476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796291c-9ad0-4156-a079-948a2f11f272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:42.444577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796291c-9ad0-4156-a079-948a2f11f272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:42.445083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796291c-9ad0-4156-a079-948a2f11f272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:42.445572 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796291c-9ad0-4156-a079-948a2f11f272 admin admin] PATH_INFO: `/v3/projects/9df149cbc6ff4b94bbcee2ac91dd5ecd` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:42.447683 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796291c-9ad0-4156-a079-948a2f11f272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df149cbc6ff4b94bbcee2ac91dd5ecd)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:42.450403 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796291c-9ad0-4156-a079-948a2f11f272 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:42.640381 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e796291c-9ad0-4156-a079-948a2f11f272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df149cbc6ff4b94bbcee2ac91dd5ecd', 'request_id': 'req-e796291c-9ad0-4156-a079-948a2f11f272'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:42.779020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e796291c-9ad0-4156-a079-948a2f11f272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df149cbc6ff4b94bbcee2ac91dd5ecd', 'request_id': 'req-e796291c-9ad0-4156-a079-948a2f11f272'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:42.788092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e796291c-9ad0-4156-a079-948a2f11f272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df149cbc6ff4b94bbcee2ac91dd5ecd', 'request_id': 'req-e796291c-9ad0-4156-a079-948a2f11f272'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:42.804707 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e796291c-9ad0-4156-a079-948a2f11f272 admin admin] The token cache is being invalidate because project 9df149cbc6ff4b94bbcee2ac91dd5ecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:50:42.805010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e796291c-9ad0-4156-a079-948a2f11f272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:42.809848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e796291c-9ad0-4156-a079-948a2f11f272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:42.809848 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1211/2436] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:50:42 2022] DELETE /identity/v3/projects/9df149cbc6ff4b94bbcee2ac91dd5ecd => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:42.813892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f90f41-2a68-4bda-b6e1-6010ad94ea7f service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:42.821847 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:42.828363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f90f41-2a68-4bda-b6e1-6010ad94ea7f service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:42.934513 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42c2c7-83a4-4746-97a1-cb6a596c5460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:42.935618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42c2c7-83a4-4746-97a1-cb6a596c5460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:42.936046 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42c2c7-83a4-4746-97a1-cb6a596c5460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:42.936453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42c2c7-83a4-4746-97a1-cb6a596c5460 admin admin] PATH_INFO: `/v3/users/a1b339b96ca4416ab12bd0eb75b48f4b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:42.937813 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42c2c7-83a4-4746-97a1-cb6a596c5460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b339b96ca4416ab12bd0eb75b48f4b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:42.940941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42c2c7-83a4-4746-97a1-cb6a596c5460 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:43.131830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a9ddb-dc6f-4404-bc3d-c7436000d456 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:43.132116 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a9ddb-dc6f-4404-bc3d-c7436000d456 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:43.132390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a9ddb-dc6f-4404-bc3d-c7436000d456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:43.153679 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a08a9ddb-dc6f-4404-bc3d-c7436000d456 None None] MFA Rules not processed for user `708d05029cef49a4a7f7cbc78fce77ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:43.154938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a08a9ddb-dc6f-4404-bc3d-c7436000d456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:43.208641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8a42c2c7-83a4-4746-97a1-cb6a596c5460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b339b96ca4416ab12bd0eb75b48f4b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:43.209396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8a42c2c7-83a4-4746-97a1-cb6a596c5460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b339b96ca4416ab12bd0eb75b48f4b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:43.216815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1212/2437] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:50:43.329881 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8a42c2c7-83a4-4746-97a1-cb6a596c5460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b339b96ca4416ab12bd0eb75b48f4b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:43.344522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8a42c2c7-83a4-4746-97a1-cb6a596c5460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b339b96ca4416ab12bd0eb75b48f4b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:43.368183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8a42c2c7-83a4-4746-97a1-cb6a596c5460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b339b96ca4416ab12bd0eb75b48f4b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:43.379513 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1226/2438] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:50:42 2022] DELETE /identity/v3/users/a1b339b96ca4416ab12bd0eb75b48f4b => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:50:43.857822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a9ddb-dc6f-4404-bc3d-c7436000d456 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:43.859792 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a08a9ddb-dc6f-4404-bc3d-c7436000d456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:43.876082 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a9ddb-dc6f-4404-bc3d-c7436000d456 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:43.911049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:43.957035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:43.957763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:43.958108 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:43.958541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] PATH_INFO: `/v3/projects/cdcff85be63649278c12d4ad16ac0384` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:43.960122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdcff85be63649278c12d4ad16ac0384)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:43.965072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:44.095488 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdcff85be63649278c12d4ad16ac0384', 'request_id': 'req-30df1b05-98db-4af1-9a9f-5580bf3b6250'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:44.198595 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdcff85be63649278c12d4ad16ac0384', 'request_id': 'req-30df1b05-98db-4af1-9a9f-5580bf3b6250'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:44.206918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdcff85be63649278c12d4ad16ac0384', 'request_id': 'req-30df1b05-98db-4af1-9a9f-5580bf3b6250'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:44.224174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] The token cache is being invalidate because project cdcff85be63649278c12d4ad16ac0384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:50:44.224545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:44.225232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:44.227025 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1213/2439] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:50:43 2022] DELETE /identity/v3/projects/cdcff85be63649278c12d4ad16ac0384 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:44.231558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42c2c7-83a4-4746-97a1-cb6a596c5460 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:44.242976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42c2c7-83a4-4746-97a1-cb6a596c5460 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:44.321125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:44.321918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:44.322565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:44.322565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] PATH_INFO: `/v3/users/c71ea606bed14e86956157d6bfa2ed10` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:44.323613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c71ea606bed14e86956157d6bfa2ed10)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:44.326096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:44.359578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:44.625303 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c71ea606bed14e86956157d6bfa2ed10'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:44.627592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c71ea606bed14e86956157d6bfa2ed10'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:44.708603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c71ea606bed14e86956157d6bfa2ed10'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:44.721896 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c71ea606bed14e86956157d6bfa2ed10'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:44.739592 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c71ea606bed14e86956157d6bfa2ed10'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:44.780496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1227/2440] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:50:44 2022] DELETE /identity/v3/users/c71ea606bed14e86956157d6bfa2ed10 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:45.375292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:45.377338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:45.390532 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df1b05-98db-4af1-9a9f-5580bf3b6250 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:45.465817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac83a4f-66b4-43e3-815b-4fb32236873d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:45.466751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac83a4f-66b4-43e3-815b-4fb32236873d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:45.467023 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac83a4f-66b4-43e3-815b-4fb32236873d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:45.467297 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac83a4f-66b4-43e3-815b-4fb32236873d admin admin] PATH_INFO: `/v3/projects/d24c465898124238b0092d58a9f9aff7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:45.468581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac83a4f-66b4-43e3-815b-4fb32236873d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d24c465898124238b0092d58a9f9aff7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:45.470614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac83a4f-66b4-43e3-815b-4fb32236873d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:45.583925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4ac83a4f-66b4-43e3-815b-4fb32236873d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd24c465898124238b0092d58a9f9aff7', 'request_id': 'req-4ac83a4f-66b4-43e3-815b-4fb32236873d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:45.687101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4ac83a4f-66b4-43e3-815b-4fb32236873d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd24c465898124238b0092d58a9f9aff7', 'request_id': 'req-4ac83a4f-66b4-43e3-815b-4fb32236873d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:45.689585 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:45.693871 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4ac83a4f-66b4-43e3-815b-4fb32236873d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd24c465898124238b0092d58a9f9aff7', 'request_id': 'req-4ac83a4f-66b4-43e3-815b-4fb32236873d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:45.705937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4ac83a4f-66b4-43e3-815b-4fb32236873d admin admin] The token cache is being invalidate because project d24c465898124238b0092d58a9f9aff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:50:45.706392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4ac83a4f-66b4-43e3-815b-4fb32236873d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:45.707053 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4ac83a4f-66b4-43e3-815b-4fb32236873d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:45.708548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1214/2441] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:50:45 2022] DELETE /identity/v3/projects/d24c465898124238b0092d58a9f9aff7 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:48.956219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:48.961561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:48.991507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef6762-1a8c-47ec-b3ec-92a09f9d415f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:49.040268 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:49.040914 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:49.041198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:49.041467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] PATH_INFO: `/v3/users/c6d30261934a429dbcc38ac0f963855d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:49.042629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6d30261934a429dbcc38ac0f963855d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:49.044662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:49.367090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6d30261934a429dbcc38ac0f963855d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:49.368003 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6d30261934a429dbcc38ac0f963855d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:49.467317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6d30261934a429dbcc38ac0f963855d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:49.475497 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6d30261934a429dbcc38ac0f963855d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:49.488581 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6d30261934a429dbcc38ac0f963855d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:49.495023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1228/2442] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:50:48 2022] DELETE /identity/v3/users/c6d30261934a429dbcc38ac0f963855d => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:49.499561 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5c4c0-8b3e-4180-acf9-3392ad8aa385 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:49.500044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5c4c0-8b3e-4180-acf9-3392ad8aa385 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:49.500128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5c4c0-8b3e-4180-acf9-3392ad8aa385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:49.520574 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4db5c4c0-8b3e-4180-acf9-3392ad8aa385 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:49.521765 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4db5c4c0-8b3e-4180-acf9-3392ad8aa385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:49.583119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1215/2443] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:49.592676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:49.594463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:49.618311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:49.657902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0fcd8-fc49-44b6-be65-b39a51139e2e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:49.662957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ef482-71c3-42f6-90b6-f3beca00f8a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:49.665635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ef482-71c3-42f6-90b6-f3beca00f8a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:49.665635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ef482-71c3-42f6-90b6-f3beca00f8a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:49.665635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ef482-71c3-42f6-90b6-f3beca00f8a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:49.674789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ef482-71c3-42f6-90b6-f3beca00f8a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:49.676258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ef482-71c3-42f6-90b6-f3beca00f8a8 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:49.686560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1229/2444] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:50.414171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5c4c0-8b3e-4180-acf9-3392ad8aa385 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:50.415777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4db5c4c0-8b3e-4180-acf9-3392ad8aa385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:50.436120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5c4c0-8b3e-4180-acf9-3392ad8aa385 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:50.442629 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416516a1-4629-4eec-8390-9724366f55a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:50.444627 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416516a1-4629-4eec-8390-9724366f55a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:50.444627 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416516a1-4629-4eec-8390-9724366f55a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:50.444627 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416516a1-4629-4eec-8390-9724366f55a4 admin admin] PATH_INFO: `/v3/projects/dab14bff5cee4e8fad0381a94aa9388f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:50.445090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416516a1-4629-4eec-8390-9724366f55a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dab14bff5cee4e8fad0381a94aa9388f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:50.447724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416516a1-4629-4eec-8390-9724366f55a4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:50.568774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-416516a1-4629-4eec-8390-9724366f55a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dab14bff5cee4e8fad0381a94aa9388f', 'request_id': 'req-416516a1-4629-4eec-8390-9724366f55a4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:50.646768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-416516a1-4629-4eec-8390-9724366f55a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dab14bff5cee4e8fad0381a94aa9388f', 'request_id': 'req-416516a1-4629-4eec-8390-9724366f55a4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:50.653841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-416516a1-4629-4eec-8390-9724366f55a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dab14bff5cee4e8fad0381a94aa9388f', 'request_id': 'req-416516a1-4629-4eec-8390-9724366f55a4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:50.671627 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-416516a1-4629-4eec-8390-9724366f55a4 admin admin] The token cache is being invalidate because project dab14bff5cee4e8fad0381a94aa9388f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:50:50.671976 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-416516a1-4629-4eec-8390-9724366f55a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:50.674417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-416516a1-4629-4eec-8390-9724366f55a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:50.676354 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1216/2445] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:50:50 2022] DELETE /identity/v3/projects/dab14bff5cee4e8fad0381a94aa9388f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:51.731789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ef482-71c3-42f6-90b6-f3beca00f8a8 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:51.733565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-040ef482-71c3-42f6-90b6-f3beca00f8a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:51.747363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ef482-71c3-42f6-90b6-f3beca00f8a8 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:51.812065 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:51.812695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:51.812970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:51.813239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] PATH_INFO: `/v3/users/4d4d8919181a46a69ae08b4a80e3bf63` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:51.814407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d4d8919181a46a69ae08b4a80e3bf63)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:51.816370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:52.095777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d4d8919181a46a69ae08b4a80e3bf63'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:52.096429 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d4d8919181a46a69ae08b4a80e3bf63'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:52.181409 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d4d8919181a46a69ae08b4a80e3bf63'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:52.189833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d4d8919181a46a69ae08b4a80e3bf63'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:52.207645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d4d8919181a46a69ae08b4a80e3bf63'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:52.214792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1230/2446] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:50:51 2022] DELETE /identity/v3/users/4d4d8919181a46a69ae08b4a80e3bf63 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:50:52.219510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd271020-92c4-4c5b-a3f9-860e88c59566 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:52.219985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd271020-92c4-4c5b-a3f9-860e88c59566 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:52.220191 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd271020-92c4-4c5b-a3f9-860e88c59566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:52.240566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bd271020-92c4-4c5b-a3f9-860e88c59566 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:52.241796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bd271020-92c4-4c5b-a3f9-860e88c59566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:52.312397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1217/2447] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:52.327978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:52.329514 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:52.382863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:52.396206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8bdfa-9533-4a45-b90b-fd5bf42ea802 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:52.400065 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60644d5a-ec6b-4ed8-978e-1bdef479f192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:52.400717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60644d5a-ec6b-4ed8-978e-1bdef479f192 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:52.401019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60644d5a-ec6b-4ed8-978e-1bdef479f192 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:52.401285 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60644d5a-ec6b-4ed8-978e-1bdef479f192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:52.412037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60644d5a-ec6b-4ed8-978e-1bdef479f192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:52.413513 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60644d5a-ec6b-4ed8-978e-1bdef479f192 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:52.424649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1231/2448] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:50:52.937971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd271020-92c4-4c5b-a3f9-860e88c59566 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:52.940602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bd271020-92c4-4c5b-a3f9-860e88c59566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:52.962710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd271020-92c4-4c5b-a3f9-860e88c59566 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:52.968623 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66462ea8-609c-4034-b71e-40182630368c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:52.969283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66462ea8-609c-4034-b71e-40182630368c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:52.969541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66462ea8-609c-4034-b71e-40182630368c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:52.969880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66462ea8-609c-4034-b71e-40182630368c admin admin] PATH_INFO: `/v3/projects/2e2f0654cfd143b282c0ae4dc98a7de4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:52.971019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66462ea8-609c-4034-b71e-40182630368c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2f0654cfd143b282c0ae4dc98a7de4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:52.972925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66462ea8-609c-4034-b71e-40182630368c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:52.982961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:53.093224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-66462ea8-609c-4034-b71e-40182630368c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2f0654cfd143b282c0ae4dc98a7de4', 'request_id': 'req-66462ea8-609c-4034-b71e-40182630368c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:53.166718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-66462ea8-609c-4034-b71e-40182630368c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2f0654cfd143b282c0ae4dc98a7de4', 'request_id': 'req-66462ea8-609c-4034-b71e-40182630368c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:53.173913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-66462ea8-609c-4034-b71e-40182630368c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2f0654cfd143b282c0ae4dc98a7de4', 'request_id': 'req-66462ea8-609c-4034-b71e-40182630368c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:53.190695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-66462ea8-609c-4034-b71e-40182630368c admin admin] The token cache is being invalidate because project 2e2f0654cfd143b282c0ae4dc98a7de4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:50:53.191060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-66462ea8-609c-4034-b71e-40182630368c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:53.191719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-66462ea8-609c-4034-b71e-40182630368c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:50:53.193497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1218/2449] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:50:52 2022] DELETE /identity/v3/projects/2e2f0654cfd143b282c0ae4dc98a7de4 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:53.206060 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef9221-e1e1-452d-bdaa-ec1a52015ba1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:53.206060 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef9221-e1e1-452d-bdaa-ec1a52015ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:53.206060 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef9221-e1e1-452d-bdaa-ec1a52015ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:53.222421 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d2ef9221-e1e1-452d-bdaa-ec1a52015ba1 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:53.223748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d2ef9221-e1e1-452d-bdaa-ec1a52015ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:53.296112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1232/2450] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:53.304625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ead9f-45ba-4ea8-98b5-e94067c32aa5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:53.305112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ead9f-45ba-4ea8-98b5-e94067c32aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:53.305313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ead9f-45ba-4ea8-98b5-e94067c32aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:53.326303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a08ead9f-45ba-4ea8-98b5-e94067c32aa5 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:53.327534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a08ead9f-45ba-4ea8-98b5-e94067c32aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:53.337811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1219/2451] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:53.342211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef9221-e1e1-452d-bdaa-ec1a52015ba1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:53.352015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef9221-e1e1-452d-bdaa-ec1a52015ba1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:53.356804 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a9c4c-362e-4236-8727-0c3aa77ce0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:53.357652 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a9c4c-362e-4236-8727-0c3aa77ce0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:53.358590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a9c4c-362e-4236-8727-0c3aa77ce0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:53.359048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a9c4c-362e-4236-8727-0c3aa77ce0b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:53.359959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a9c4c-362e-4236-8727-0c3aa77ce0b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:50:53.360262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a9c4c-362e-4236-8727-0c3aa77ce0b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:53.362772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a9c4c-362e-4236-8727-0c3aa77ce0b2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:53.377216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1233/2452] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:50:53.381077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ead9f-45ba-4ea8-98b5-e94067c32aa5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:53.391155 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ead9f-45ba-4ea8-98b5-e94067c32aa5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:53.396484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda58968-a8a8-475e-960f-b24bfbd2edf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:53.397107 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda58968-a8a8-475e-960f-b24bfbd2edf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:53.397378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda58968-a8a8-475e-960f-b24bfbd2edf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:53.397639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda58968-a8a8-475e-960f-b24bfbd2edf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:53.398324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda58968-a8a8-475e-960f-b24bfbd2edf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:53.400366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda58968-a8a8-475e-960f-b24bfbd2edf3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:53.523824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1220/2453] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:50:53 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:50:53.528153 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a9c4c-362e-4236-8727-0c3aa77ce0b2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:53.539125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a9c4c-362e-4236-8727-0c3aa77ce0b2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:53.597540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf792c-51d0-46db-9a62-8703c05389e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:53.598171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf792c-51d0-46db-9a62-8703c05389e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:53.598462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf792c-51d0-46db-9a62-8703c05389e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:53.598735 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf792c-51d0-46db-9a62-8703c05389e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:53.599362 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf792c-51d0-46db-9a62-8703c05389e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:53.601340 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf792c-51d0-46db-9a62-8703c05389e2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:53.717780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1234/2454] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:50:53 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:50:53.722544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda58968-a8a8-475e-960f-b24bfbd2edf3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:53.734365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda58968-a8a8-475e-960f-b24bfbd2edf3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:53.740018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345360b-985f-46e8-a008-bd8a018e2af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:53.740624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345360b-985f-46e8-a008-bd8a018e2af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:53.740901 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345360b-985f-46e8-a008-bd8a018e2af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:53.741167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345360b-985f-46e8-a008-bd8a018e2af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:53.741841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345360b-985f-46e8-a008-bd8a018e2af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:53.743491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345360b-985f-46e8-a008-bd8a018e2af6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:53.754468 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1221/2455] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:50:53.758541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf792c-51d0-46db-9a62-8703c05389e2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:53.770469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf792c-51d0-46db-9a62-8703c05389e2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:53.775318 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa654c-a4ae-4a97-bfe2-312d485d91d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:53.775864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa654c-a4ae-4a97-bfe2-312d485d91d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:53.776145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa654c-a4ae-4a97-bfe2-312d485d91d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:53.776415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa654c-a4ae-4a97-bfe2-312d485d91d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:53.777058 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa654c-a4ae-4a97-bfe2-312d485d91d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:53.778728 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa654c-a4ae-4a97-bfe2-312d485d91d9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:53.792201 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1235/2456] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:50:53.796385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345360b-985f-46e8-a008-bd8a018e2af6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:53.806903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345360b-985f-46e8-a008-bd8a018e2af6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:53.811962 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c6fa5-85e1-4c1a-bc60-dd47b801bb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:53.812675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c6fa5-85e1-4c1a-bc60-dd47b801bb00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:53.812949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c6fa5-85e1-4c1a-bc60-dd47b801bb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:53.813217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c6fa5-85e1-4c1a-bc60-dd47b801bb00 admin admin] PATH_INFO: `/v3/projects/ed372e1fa4c348b79d47cedd8f818382/users/f39db9c569ef4fe19357b7af7033321d/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:53.833114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c6fa5-85e1-4c1a-bc60-dd47b801bb00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed372e1fa4c348b79d47cedd8f818382, user_id=f39db9c569ef4fe19357b7af7033321d, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:53.835875 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c6fa5-85e1-4c1a-bc60-dd47b801bb00 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:54.002007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1222/2457] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:50:53 2022] PUT /identity/v3/projects/ed372e1fa4c348b79d47cedd8f818382/users/f39db9c569ef4fe19357b7af7033321d/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:50:54.013458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1a184-fee4-4b20-8d39-3eff663045ba None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:54.013758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1a184-fee4-4b20-8d39-3eff663045ba None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:54.014158 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1a184-fee4-4b20-8d39-3eff663045ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:54.047103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b9e1a184-fee4-4b20-8d39-3eff663045ba None None] MFA Rules not processed for user `f39db9c569ef4fe19357b7af7033321d`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:50:54.048204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b9e1a184-fee4-4b20-8d39-3eff663045ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:54.089122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:50:54.114608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1236/2458] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:50:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:50:54.127002 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c6fa5-85e1-4c1a-bc60-dd47b801bb00 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:50:54.128775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6a1c6fa5-85e1-4c1a-bc60-dd47b801bb00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:50:54.187166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c6fa5-85e1-4c1a-bc60-dd47b801bb00 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:50:54.191778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9600726-e699-49e2-b2a0-cff5c2964628 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b86978df61245ae88fead154193fcd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:50:54.192404 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9600726-e699-49e2-b2a0-cff5c2964628 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:50:54.192681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9600726-e699-49e2-b2a0-cff5c2964628 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:50:54.192952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9600726-e699-49e2-b2a0-cff5c2964628 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:50:54.201926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9600726-e699-49e2-b2a0-cff5c2964628 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:50:54.203413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9600726-e699-49e2-b2a0-cff5c2964628 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:50:54.213205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1223/2459] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:50:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:51:04.169889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=9 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:51:04.209046 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:51:30.447294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1a184-fee4-4b20-8d39-3eff663045ba None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:30.521808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1a184-fee4-4b20-8d39-3eff663045ba None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:30.527232 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:30.527874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:30.528144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:30.528434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] PATH_INFO: `/v3/users/f39db9c569ef4fe19357b7af7033321d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:30.529561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f39db9c569ef4fe19357b7af7033321d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:30.531536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:30.728444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f39db9c569ef4fe19357b7af7033321d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:51:30.729192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f39db9c569ef4fe19357b7af7033321d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:51:30.788841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f39db9c569ef4fe19357b7af7033321d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:51:30.797644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f39db9c569ef4fe19357b7af7033321d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:51:30.812348 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f39db9c569ef4fe19357b7af7033321d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:51:30.819961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1237/2460] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:51:30 2022] DELETE /identity/v3/users/f39db9c569ef4fe19357b7af7033321d => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:51:30.824275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1274de2-2d41-425a-80fe-8b2d072ee314 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:30.824768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1274de2-2d41-425a-80fe-8b2d072ee314 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:30.824869 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1274de2-2d41-425a-80fe-8b2d072ee314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:30.844371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c1274de2-2d41-425a-80fe-8b2d072ee314 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:51:30.846111 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c1274de2-2d41-425a-80fe-8b2d072ee314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:30.910803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1224/2461] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:51:30.920772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:30.922271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:30.983181 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57a182-6859-4aa2-afac-7b0418d2fb23 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:30.987901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbce327-6538-48ba-b00d-e7c26d6dba28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:30.988507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbce327-6538-48ba-b00d-e7c26d6dba28 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:30.988784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbce327-6538-48ba-b00d-e7c26d6dba28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:30.989053 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbce327-6538-48ba-b00d-e7c26d6dba28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:30.998852 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbce327-6538-48ba-b00d-e7c26d6dba28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:31.000268 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbce327-6538-48ba-b00d-e7c26d6dba28 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:31.010225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1238/2462] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:51:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:51:31.488554 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1274de2-2d41-425a-80fe-8b2d072ee314 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:31.490317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c1274de2-2d41-425a-80fe-8b2d072ee314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:31.505937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1274de2-2d41-425a-80fe-8b2d072ee314 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:31.512192 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447822dc-c2fc-408b-ab9e-26cc2cf43afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:31.512729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447822dc-c2fc-408b-ab9e-26cc2cf43afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:31.513212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447822dc-c2fc-408b-ab9e-26cc2cf43afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:31.513212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447822dc-c2fc-408b-ab9e-26cc2cf43afb admin admin] PATH_INFO: `/v3/projects/ed372e1fa4c348b79d47cedd8f818382` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:31.514413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447822dc-c2fc-408b-ab9e-26cc2cf43afb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed372e1fa4c348b79d47cedd8f818382)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:31.516188 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447822dc-c2fc-408b-ab9e-26cc2cf43afb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:31.628655 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-447822dc-c2fc-408b-ab9e-26cc2cf43afb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed372e1fa4c348b79d47cedd8f818382', 'request_id': 'req-447822dc-c2fc-408b-ab9e-26cc2cf43afb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:51:31.700774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-447822dc-c2fc-408b-ab9e-26cc2cf43afb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed372e1fa4c348b79d47cedd8f818382', 'request_id': 'req-447822dc-c2fc-408b-ab9e-26cc2cf43afb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:51:31.708428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-447822dc-c2fc-408b-ab9e-26cc2cf43afb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed372e1fa4c348b79d47cedd8f818382', 'request_id': 'req-447822dc-c2fc-408b-ab9e-26cc2cf43afb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:51:31.721317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-447822dc-c2fc-408b-ab9e-26cc2cf43afb admin admin] The token cache is being invalidate because project ed372e1fa4c348b79d47cedd8f818382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:51:31.721317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-447822dc-c2fc-408b-ab9e-26cc2cf43afb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:51:31.721903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-447822dc-c2fc-408b-ab9e-26cc2cf43afb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:51:31.723442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1225/2463] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:51:31 2022] DELETE /identity/v3/projects/ed372e1fa4c348b79d47cedd8f818382 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:51:31.730788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87dcf60-693e-444c-b659-c3d4f7c30fac None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:31.731494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87dcf60-693e-444c-b659-c3d4f7c30fac None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:31.731494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87dcf60-693e-444c-b659-c3d4f7c30fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:31.754086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b87dcf60-693e-444c-b659-c3d4f7c30fac None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:51:31.755144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b87dcf60-693e-444c-b659-c3d4f7c30fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:31.774879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:51:31.844280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1239/2464] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:51:31.852520 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72486fa-1028-4ab7-b531-1cc27d49c401 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:31.853169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72486fa-1028-4ab7-b531-1cc27d49c401 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:31.853169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72486fa-1028-4ab7-b531-1cc27d49c401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:31.880976 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c72486fa-1028-4ab7-b531-1cc27d49c401 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:51:31.882291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c72486fa-1028-4ab7-b531-1cc27d49c401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:31.890964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1226/2465] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:51:31.895999 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87dcf60-693e-444c-b659-c3d4f7c30fac None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:31.906616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87dcf60-693e-444c-b659-c3d4f7c30fac None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:31.912265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb4fbb-1aff-47c3-a1d8-4c90a86acf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:31.913041 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb4fbb-1aff-47c3-a1d8-4c90a86acf01 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:31.913381 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb4fbb-1aff-47c3-a1d8-4c90a86acf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:31.913712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb4fbb-1aff-47c3-a1d8-4c90a86acf01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:31.914409 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb4fbb-1aff-47c3-a1d8-4c90a86acf01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:51:31.915130 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb4fbb-1aff-47c3-a1d8-4c90a86acf01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:31.916377 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb4fbb-1aff-47c3-a1d8-4c90a86acf01 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:31.930397 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1240/2466] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:51:31.935050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72486fa-1028-4ab7-b531-1cc27d49c401 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:31.948286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72486fa-1028-4ab7-b531-1cc27d49c401 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:31.955914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da22411-c47f-41f7-a708-6ef18aa11361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:31.956537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da22411-c47f-41f7-a708-6ef18aa11361 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:31.956887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da22411-c47f-41f7-a708-6ef18aa11361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:31.957133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da22411-c47f-41f7-a708-6ef18aa11361 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:31.957777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da22411-c47f-41f7-a708-6ef18aa11361 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:31.960407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da22411-c47f-41f7-a708-6ef18aa11361 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:32.062708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb4fbb-1aff-47c3-a1d8-4c90a86acf01 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:32.064482 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-25bb4fbb-1aff-47c3-a1d8-4c90a86acf01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:32.108771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1227/2467] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:51:31 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:51:32.109873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da22411-c47f-41f7-a708-6ef18aa11361 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:32.129249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb4fbb-1aff-47c3-a1d8-4c90a86acf01 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:32.178378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:51:32.183770 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da22411-c47f-41f7-a708-6ef18aa11361 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:32.185258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e04e4-a270-4738-9806-f60fb0591c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:32.186011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e04e4-a270-4738-9806-f60fb0591c88 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:32.186322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e04e4-a270-4738-9806-f60fb0591c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:32.193559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e04e4-a270-4738-9806-f60fb0591c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:32.193559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a22e04e4-a270-4738-9806-f60fb0591c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:32.193795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46431747-2b34-421d-bc2c-32900acf52dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:32.193795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46431747-2b34-421d-bc2c-32900acf52dd admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:32.193795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46431747-2b34-421d-bc2c-32900acf52dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:32.193795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46431747-2b34-421d-bc2c-32900acf52dd admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:32.194183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46431747-2b34-421d-bc2c-32900acf52dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:32.194183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46431747-2b34-421d-bc2c-32900acf52dd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:32.264855 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e04e4-a270-4738-9806-f60fb0591c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:32.266382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e04e4-a270-4738-9806-f60fb0591c88 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:32.280104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1228/2468] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:51:32 2022] POST /identity/v3/users => generated 291 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:51:32.281208 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1241/2469] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:51:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:51:32.284367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46431747-2b34-421d-bc2c-32900acf52dd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:32.301268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46431747-2b34-421d-bc2c-32900acf52dd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:32.307715 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d774a-024f-4031-b94b-a4763edad1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:32.308535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d774a-024f-4031-b94b-a4763edad1fd admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:32.308855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d774a-024f-4031-b94b-a4763edad1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:32.309136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d774a-024f-4031-b94b-a4763edad1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:32.309790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d774a-024f-4031-b94b-a4763edad1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:32.311501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d774a-024f-4031-b94b-a4763edad1fd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:32.324131 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1229/2470] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:51:32.328262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e04e4-a270-4738-9806-f60fb0591c88 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:32.341210 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e04e4-a270-4738-9806-f60fb0591c88 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:32.347463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d43d8-eb5c-4e43-a435-22655009d596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:32.348081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d43d8-eb5c-4e43-a435-22655009d596 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:32.348365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d43d8-eb5c-4e43-a435-22655009d596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:32.348784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d43d8-eb5c-4e43-a435-22655009d596 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:32.349570 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d43d8-eb5c-4e43-a435-22655009d596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:32.352229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d43d8-eb5c-4e43-a435-22655009d596 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:32.366227 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1242/2471] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:51:32.371101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d774a-024f-4031-b94b-a4763edad1fd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:32.386933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d774a-024f-4031-b94b-a4763edad1fd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:32.393421 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7517ae3-f413-4cef-827c-8590c07c7fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:32.394103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7517ae3-f413-4cef-827c-8590c07c7fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:32.394405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7517ae3-f413-4cef-827c-8590c07c7fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:32.394676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7517ae3-f413-4cef-827c-8590c07c7fd5 admin admin] PATH_INFO: `/v3/projects/94ceae42bd11404eaea28d523b98126e/users/bae09cfbf3844b1682d8924e7d9578ee/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:32.413222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7517ae3-f413-4cef-827c-8590c07c7fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ceae42bd11404eaea28d523b98126e, user_id=bae09cfbf3844b1682d8924e7d9578ee, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:32.416135 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7517ae3-f413-4cef-827c-8590c07c7fd5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:32.526829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1230/2472] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:51:32 2022] PUT /identity/v3/projects/94ceae42bd11404eaea28d523b98126e/users/bae09cfbf3844b1682d8924e7d9578ee/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:51:32.532015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56df37-4046-4ad8-8b8f-7a792d604f1c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:32.532315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56df37-4046-4ad8-8b8f-7a792d604f1c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:32.532862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56df37-4046-4ad8-8b8f-7a792d604f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:32.544693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:51:32.557172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1f56df37-4046-4ad8-8b8f-7a792d604f1c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:51:32.559260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1f56df37-4046-4ad8-8b8f-7a792d604f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:32.631683 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1243/2473] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:51:32.651620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7517ae3-f413-4cef-827c-8590c07c7fd5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:32.736479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7517ae3-f413-4cef-827c-8590c07c7fd5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:32.740417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9470c9d-e7e1-42a2-b619-ed6e669ad6ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:32.741034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9470c9d-e7e1-42a2-b619-ed6e669ad6ee service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:32.741307 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9470c9d-e7e1-42a2-b619-ed6e669ad6ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:32.741570 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9470c9d-e7e1-42a2-b619-ed6e669ad6ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:32.751493 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9470c9d-e7e1-42a2-b619-ed6e669ad6ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:32.753120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9470c9d-e7e1-42a2-b619-ed6e669ad6ee service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:32.765125 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1231/2474] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:51:35.123092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a887f57-2330-41f5-a6b6-1654df68ec90 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:35.123092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a887f57-2330-41f5-a6b6-1654df68ec90 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:35.123782 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a887f57-2330-41f5-a6b6-1654df68ec90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:35.156149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8a887f57-2330-41f5-a6b6-1654df68ec90 None None] MFA Rules not processed for user `bae09cfbf3844b1682d8924e7d9578ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:51:35.158032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8a887f57-2330-41f5-a6b6-1654df68ec90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:35.236435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1244/2475] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:51:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:51:35.249039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9470c9d-e7e1-42a2-b619-ed6e669ad6ee service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:35.250930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c9470c9d-e7e1-42a2-b619-ed6e669ad6ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:35.331547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9470c9d-e7e1-42a2-b619-ed6e669ad6ee service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:35.337075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542059e4-b580-4b63-b1ef-920945a54345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:35.338981 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542059e4-b580-4b63-b1ef-920945a54345 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:35.339336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542059e4-b580-4b63-b1ef-920945a54345 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:35.339697 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542059e4-b580-4b63-b1ef-920945a54345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:35.354166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542059e4-b580-4b63-b1ef-920945a54345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:35.355860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542059e4-b580-4b63-b1ef-920945a54345 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:35.367019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1232/2476] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:51:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:51:37.274537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf1105-ca41-431d-99a4-247c1b60a86b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:37.275415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf1105-ca41-431d-99a4-247c1b60a86b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:37.275415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf1105-ca41-431d-99a4-247c1b60a86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:37.298395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5baf1105-ca41-431d-99a4-247c1b60a86b None None] MFA Rules not processed for user `bae09cfbf3844b1682d8924e7d9578ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:51:37.299602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5baf1105-ca41-431d-99a4-247c1b60a86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:37.306911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1245/2477] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:51:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:51:37.347322 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542059e4-b580-4b63-b1ef-920945a54345 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:37.348764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-542059e4-b580-4b63-b1ef-920945a54345 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:37.407606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542059e4-b580-4b63-b1ef-920945a54345 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:37.411128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f63aa9-b424-4735-82a4-1df91af70373 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b86978df61245ae88fead154193fcd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:37.411735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f63aa9-b424-4735-82a4-1df91af70373 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:37.412016 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f63aa9-b424-4735-82a4-1df91af70373 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:37.412284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f63aa9-b424-4735-82a4-1df91af70373 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:37.413996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-76f63aa9-b424-4735-82a4-1df91af70373 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:37.424058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:51:37.487114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f63aa9-b424-4735-82a4-1df91af70373 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:37.488541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f63aa9-b424-4735-82a4-1df91af70373 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:37.500042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1233/2478] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:51:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:51:37.603138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf1105-ca41-431d-99a4-247c1b60a86b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:37.612902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf1105-ca41-431d-99a4-247c1b60a86b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:37.616409 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9c366-f12b-4196-9516-e4dd717c48ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:37.617006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9c366-f12b-4196-9516-e4dd717c48ca service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:37.617275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9c366-f12b-4196-9516-e4dd717c48ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:37.617532 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9c366-f12b-4196-9516-e4dd717c48ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:37.619263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-82d9c366-f12b-4196-9516-e4dd717c48ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:37.633542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9c366-f12b-4196-9516-e4dd717c48ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:37.634991 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9c366-f12b-4196-9516-e4dd717c48ca service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:37.646848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1246/2479] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:51:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:51:47.495565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=29 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:51:47.642610 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=66 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:51:55.689414 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f63aa9-b424-4735-82a4-1df91af70373 service glance] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:55.699790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f63aa9-b424-4735-82a4-1df91af70373 service glance] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:55.705548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3613788-e3d6-415e-afb5-269af2d49407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:55.706467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3613788-e3d6-415e-afb5-269af2d49407 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:55.706467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3613788-e3d6-415e-afb5-269af2d49407 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:55.706727 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3613788-e3d6-415e-afb5-269af2d49407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:55.708563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c3613788-e3d6-415e-afb5-269af2d49407 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:55.829539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3613788-e3d6-415e-afb5-269af2d49407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:55.849199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3613788-e3d6-415e-afb5-269af2d49407 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:55.849199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1234/2480] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:51:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:51:56.826197 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9c366-f12b-4196-9516-e4dd717c48ca service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:56.828241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-82d9c366-f12b-4196-9516-e4dd717c48ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:56.852641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9c366-f12b-4196-9516-e4dd717c48ca service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:51:56.885417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b471378-cfa7-4944-bcd2-a43492af7154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:51:56.886077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b471378-cfa7-4944-bcd2-a43492af7154 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:56.886405 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b471378-cfa7-4944-bcd2-a43492af7154 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:56.886676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b471378-cfa7-4944-bcd2-a43492af7154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:56.888591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0b471378-cfa7-4944-bcd2-a43492af7154 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:56.964851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b471378-cfa7-4944-bcd2-a43492af7154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:51:56.966525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b471378-cfa7-4944-bcd2-a43492af7154 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:51:56.977408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1247/2481] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:51:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:51:58.560245 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67e3fb-37cf-4512-ace8-2ac9836593df None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:58.561113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67e3fb-37cf-4512-ace8-2ac9836593df None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:58.561634 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67e3fb-37cf-4512-ace8-2ac9836593df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:58.590964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6f67e3fb-37cf-4512-ace8-2ac9836593df None None] MFA Rules not processed for user `2aea4a0e0ec74c538735b890fb774b14`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:51:58.592168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6f67e3fb-37cf-4512-ace8-2ac9836593df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:58.606287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1235/2482] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:51:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:51:59.667822 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445d419-f629-42aa-9f79-f7526d6fc012 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:51:59.668435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445d419-f629-42aa-9f79-f7526d6fc012 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:51:59.668435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445d419-f629-42aa-9f79-f7526d6fc012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:51:59.691518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2445d419-f629-42aa-9f79-f7526d6fc012 None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:51:59.694371 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2445d419-f629-42aa-9f79-f7526d6fc012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:59.757648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1248/2483] 10.222.0.44 () {58 vars in 967 bytes} [Sun Sep 11 08:51:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:51:59.913329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67e3fb-37cf-4512-ace8-2ac9836593df None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:51:59.914838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6f67e3fb-37cf-4512-ace8-2ac9836593df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:51:59.998468 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67e3fb-37cf-4512-ace8-2ac9836593df None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:52:00.003915 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79ee13-e91e-4331-81b6-ec810d5ec32d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:52:00.004580 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79ee13-e91e-4331-81b6-ec810d5ec32d service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:00.004893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79ee13-e91e-4331-81b6-ec810d5ec32d service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:00.005367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79ee13-e91e-4331-81b6-ec810d5ec32d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:00.023594 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79ee13-e91e-4331-81b6-ec810d5ec32d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:52:00.025504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79ee13-e91e-4331-81b6-ec810d5ec32d service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:52:00.045791 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1236/2484] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:51:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:52:04.863559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445d419-f629-42aa-9f79-f7526d6fc012 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:52:04.865763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2445d419-f629-42aa-9f79-f7526d6fc012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:04.889491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445d419-f629-42aa-9f79-f7526d6fc012 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:52:04.902297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1d99a-013c-4733-9d68-0c84cf6e991c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:52:04.909663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1d99a-013c-4733-9d68-0c84cf6e991c service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:04.909663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1d99a-013c-4733-9d68-0c84cf6e991c service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:04.909663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1d99a-013c-4733-9d68-0c84cf6e991c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:04.909663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ddb1d99a-013c-4733-9d68-0c84cf6e991c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:05.026306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1d99a-013c-4733-9d68-0c84cf6e991c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:52:05.027789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1d99a-013c-4733-9d68-0c84cf6e991c service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:52:05.037828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1249/2485] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:52:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:52:06.515638 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb1dfd-1b0d-4639-9486-53eab558548b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:06.516171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb1dfd-1b0d-4639-9486-53eab558548b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:06.516255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb1dfd-1b0d-4639-9486-53eab558548b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:06.536374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f6cb1dfd-1b0d-4639-9486-53eab558548b None None] MFA Rules not processed for user `64ea6fbc6e7d42dabdf459e8de4c3621`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:52:06.537513 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6cb1dfd-1b0d-4639-9486-53eab558548b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:06.544432 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1237/2486] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:52:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:52:06.865869 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1d99a-013c-4733-9d68-0c84cf6e991c service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:52:06.880297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1d99a-013c-4733-9d68-0c84cf6e991c service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:52:06.904066 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c547f1-4e7c-4a25-a0ee-c8bfecffc8ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:52:06.904704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c547f1-4e7c-4a25-a0ee-c8bfecffc8ac service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:06.905048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c547f1-4e7c-4a25-a0ee-c8bfecffc8ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:06.905250 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c547f1-4e7c-4a25-a0ee-c8bfecffc8ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:06.907667 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-04c547f1-4e7c-4a25-a0ee-c8bfecffc8ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:06.919261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c547f1-4e7c-4a25-a0ee-c8bfecffc8ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:52:06.922392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c547f1-4e7c-4a25-a0ee-c8bfecffc8ac service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:52:06.931546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1250/2487] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:52:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:52:07.402768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb1dfd-1b0d-4639-9486-53eab558548b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:52:07.404246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6cb1dfd-1b0d-4639-9486-53eab558548b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:07.512957 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb1dfd-1b0d-4639-9486-53eab558548b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:52:07.518189 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc626c47-f6da-4a60-91ea-43b5cb269a1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d03db170fb6460794e32669456bcdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:52:07.518839 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc626c47-f6da-4a60-91ea-43b5cb269a1e service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:07.519068 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc626c47-f6da-4a60-91ea-43b5cb269a1e service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:07.519337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc626c47-f6da-4a60-91ea-43b5cb269a1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:07.521092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bc626c47-f6da-4a60-91ea-43b5cb269a1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:07.534546 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc626c47-f6da-4a60-91ea-43b5cb269a1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:52:07.536145 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc626c47-f6da-4a60-91ea-43b5cb269a1e service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:52:07.548134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1238/2488] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:52:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:52:11.003625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c547f1-4e7c-4a25-a0ee-c8bfecffc8ac service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:52:11.008622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-04c547f1-4e7c-4a25-a0ee-c8bfecffc8ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:11.035243 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c547f1-4e7c-4a25-a0ee-c8bfecffc8ac service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:52:11.041501 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597b4b7-eae8-4b14-ab09-e9e4a345ab3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d03db170fb6460794e32669456bcdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:52:11.042137 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597b4b7-eae8-4b14-ab09-e9e4a345ab3b service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:11.042489 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597b4b7-eae8-4b14-ab09-e9e4a345ab3b service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:11.042762 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597b4b7-eae8-4b14-ab09-e9e4a345ab3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:11.044959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e597b4b7-eae8-4b14-ab09-e9e4a345ab3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:11.059672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597b4b7-eae8-4b14-ab09-e9e4a345ab3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:52:11.061709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597b4b7-eae8-4b14-ab09-e9e4a345ab3b service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:52:11.085637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1251/2489] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:52:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:52:11.168168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc626c47-f6da-4a60-91ea-43b5cb269a1e service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:52:11.169873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bc626c47-f6da-4a60-91ea-43b5cb269a1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:11.184039 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:52:11.190586 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc626c47-f6da-4a60-91ea-43b5cb269a1e service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:52:11.196578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:52:11.199396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:11.200136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:11.200657 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] PATH_INFO: `/v3/users/a0dde6aa81204252b66b6e640b343ac3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:11.202320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0dde6aa81204252b66b6e640b343ac3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:52:11.206516 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:52:11.561796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0dde6aa81204252b66b6e640b343ac3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:11.584154 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0dde6aa81204252b66b6e640b343ac3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:11.724563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0dde6aa81204252b66b6e640b343ac3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:11.724563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0dde6aa81204252b66b6e640b343ac3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:11.747757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0dde6aa81204252b66b6e640b343ac3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:11.756351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1239/2490] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:52:11 2022] DELETE /identity/v3/users/a0dde6aa81204252b66b6e640b343ac3 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:52:12.043699 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597b4b7-eae8-4b14-ab09-e9e4a345ab3b service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:52:12.059185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597b4b7-eae8-4b14-ab09-e9e4a345ab3b service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:52:12.086964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:52:12.149992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:52:12.150652 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:12.150929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:12.151198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] PATH_INFO: `/v3/projects/b4f426194c8d41bcaee7862c5d04b180` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:12.152330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4f426194c8d41bcaee7862c5d04b180)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:52:12.154346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:52:12.325409 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4f426194c8d41bcaee7862c5d04b180', 'request_id': 'req-b26f150d-e068-487f-a551-97d754c8222c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:12.409119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4f426194c8d41bcaee7862c5d04b180', 'request_id': 'req-b26f150d-e068-487f-a551-97d754c8222c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:12.416159 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4f426194c8d41bcaee7862c5d04b180', 'request_id': 'req-b26f150d-e068-487f-a551-97d754c8222c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:12.430374 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] The token cache is being invalidate because project b4f426194c8d41bcaee7862c5d04b180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:52:12.430925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:12.431815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:12.440153 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1252/2491] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:52:12 2022] DELETE /identity/v3/projects/b4f426194c8d41bcaee7862c5d04b180 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:52:12.444982 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:52:12.446644 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:12.462217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b632f4-3768-4db8-9876-de0455a4cfd9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:52:12.540711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:52:12.541640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:12.542087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:12.542654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] PATH_INFO: `/v3/users/0c4402e367804da6a1ce5b977b52b760` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:12.544595 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c4402e367804da6a1ce5b977b52b760)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:52:12.546652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:52:12.801803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c4402e367804da6a1ce5b977b52b760'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:12.820486 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c4402e367804da6a1ce5b977b52b760'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:12.894457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:52:12.895984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:12.898410 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c4402e367804da6a1ce5b977b52b760'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:12.899006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c4402e367804da6a1ce5b977b52b760'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:12.918995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:52:12.924050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c4402e367804da6a1ce5b977b52b760'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:12.952067 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1240/2492] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:52:12 2022] DELETE /identity/v3/users/0c4402e367804da6a1ce5b977b52b760 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:52:12.977709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f150d-e068-487f-a551-97d754c8222c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:52:13.049577 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788adcd-44bc-4930-adec-dc8e6fc46951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:52:13.050975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788adcd-44bc-4930-adec-dc8e6fc46951 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:13.051429 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788adcd-44bc-4930-adec-dc8e6fc46951 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:13.051904 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788adcd-44bc-4930-adec-dc8e6fc46951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:13.054415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4788adcd-44bc-4930-adec-dc8e6fc46951 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:13.067360 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788adcd-44bc-4930-adec-dc8e6fc46951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:52:13.069342 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788adcd-44bc-4930-adec-dc8e6fc46951 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:52:13.078657 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1253/2493] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:52:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:52:13.311078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:52:13.313440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:13.403938 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3db367-3e55-41e1-ae4f-182ff2071b0c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:52:13.409209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:52:13.409870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:13.410139 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:13.410652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] PATH_INFO: `/v3/projects/8125bc49eda643a282501e1ada9fe4f9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:13.412011 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8125bc49eda643a282501e1ada9fe4f9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:52:13.415464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:52:13.570419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8125bc49eda643a282501e1ada9fe4f9', 'request_id': 'req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:13.668908 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8125bc49eda643a282501e1ada9fe4f9', 'request_id': 'req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:13.682188 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8125bc49eda643a282501e1ada9fe4f9', 'request_id': 'req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:13.700558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] The token cache is being invalidate because project 8125bc49eda643a282501e1ada9fe4f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:52:13.701095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:13.702072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:52:13.704631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1241/2494] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:52:13 2022] DELETE /identity/v3/projects/8125bc49eda643a282501e1ada9fe4f9 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:52:16.137937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ef4e6-51d2-4531-906a-3819bff00f7e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:16.138581 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ef4e6-51d2-4531-906a-3819bff00f7e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:16.138581 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ef4e6-51d2-4531-906a-3819bff00f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:16.147950 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:52:16.161903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-262ef4e6-51d2-4531-906a-3819bff00f7e None None] MFA Rules not processed for user `3abba619decf448db477533c663004bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:52:16.163325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-262ef4e6-51d2-4531-906a-3819bff00f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:16.231926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1254/2495] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:52:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:52:16.545694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:52:16.547649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:16.603434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4d8c9-1e9c-46e7-ab5c-c6b5d915ce03 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:52:16.606958 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d441d-b411-4078-b0b4-91965e0ce628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:52:16.607557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d441d-b411-4078-b0b4-91965e0ce628 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:16.607835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d441d-b411-4078-b0b4-91965e0ce628 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:16.608105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d441d-b411-4078-b0b4-91965e0ce628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:16.610736 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa8d441d-b411-4078-b0b4-91965e0ce628 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:16.620226 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d441d-b411-4078-b0b4-91965e0ce628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:52:16.621752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d441d-b411-4078-b0b4-91965e0ce628 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:52:16.629711 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1242/2496] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:52:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:52:24.739947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ef4e6-51d2-4531-906a-3819bff00f7e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:52:24.741364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-262ef4e6-51d2-4531-906a-3819bff00f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:24.833541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ef4e6-51d2-4531-906a-3819bff00f7e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:52:24.839450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a5dd8-b772-43bd-b0b5-816364549baa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:52:24.840081 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a5dd8-b772-43bd-b0b5-816364549baa service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:24.846966 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a5dd8-b772-43bd-b0b5-816364549baa service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:24.846966 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a5dd8-b772-43bd-b0b5-816364549baa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:24.846966 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-729a5dd8-b772-43bd-b0b5-816364549baa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:24.855316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a5dd8-b772-43bd-b0b5-816364549baa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:52:24.856955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a5dd8-b772-43bd-b0b5-816364549baa service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:52:24.874709 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1255/2497] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:52:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:52:26.626243 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:52:34.864483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=57 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:52:43.068780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d441d-b411-4078-b0b4-91965e0ce628 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:52:43.097791 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d441d-b411-4078-b0b4-91965e0ce628 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:52:43.111087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b400a-7766-46c7-9d38-b242f056a55d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:52:43.111758 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b400a-7766-46c7-9d38-b242f056a55d service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:43.112035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b400a-7766-46c7-9d38-b242f056a55d service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:43.112335 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b400a-7766-46c7-9d38-b242f056a55d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:43.123117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-884b400a-7766-46c7-9d38-b242f056a55d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:43.139437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b400a-7766-46c7-9d38-b242f056a55d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:52:43.141726 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b400a-7766-46c7-9d38-b242f056a55d service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:52:43.155440 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1243/2498] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:52:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:52:53.149406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:52:55.693628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d2e09-cca1-4786-891d-4a3901da6382 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:52:55.694498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d2e09-cca1-4786-891d-4a3901da6382 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:52:55.694498 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d2e09-cca1-4786-891d-4a3901da6382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:52:55.717766 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d59d2e09-cca1-4786-891d-4a3901da6382 None None] MFA Rules not processed for user `bae09cfbf3844b1682d8924e7d9578ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:52:55.719153 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d59d2e09-cca1-4786-891d-4a3901da6382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:52:55.801351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1256/2499] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:52:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:53:05.785507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=25 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:53:08.078188 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b400a-7766-46c7-9d38-b242f056a55d service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:08.090544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b400a-7766-46c7-9d38-b242f056a55d service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:08.096836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96c98e-79fd-4482-b028-4f4ddcf08f12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:08.104128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96c98e-79fd-4482-b028-4f4ddcf08f12 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:08.104128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96c98e-79fd-4482-b028-4f4ddcf08f12 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:08.104128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96c98e-79fd-4482-b028-4f4ddcf08f12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:08.104128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8a96c98e-79fd-4482-b028-4f4ddcf08f12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:08.113752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96c98e-79fd-4482-b028-4f4ddcf08f12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:08.115966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96c98e-79fd-4482-b028-4f4ddcf08f12 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:08.127638 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1244/2500] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:53:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:53:16.455346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d2e09-cca1-4786-891d-4a3901da6382 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:16.456801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d59d2e09-cca1-4786-891d-4a3901da6382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:16.469493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d2e09-cca1-4786-891d-4a3901da6382 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:16.473306 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a8099-abf0-44df-a656-61670025e016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:16.473906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a8099-abf0-44df-a656-61670025e016 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:16.474206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a8099-abf0-44df-a656-61670025e016 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:16.474476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a8099-abf0-44df-a656-61670025e016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:16.476178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ac4a8099-abf0-44df-a656-61670025e016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:16.542493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a8099-abf0-44df-a656-61670025e016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:16.543950 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a8099-abf0-44df-a656-61670025e016 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:16.553951 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1257/2501] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:53:18.122257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:53:20.702604 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96c98e-79fd-4482-b028-4f4ddcf08f12 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:20.704714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8a96c98e-79fd-4482-b028-4f4ddcf08f12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:20.719804 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96c98e-79fd-4482-b028-4f4ddcf08f12 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:20.724391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be14367-d947-472a-b24c-c2a1e7183e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:20.725019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be14367-d947-472a-b24c-c2a1e7183e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:20.725276 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be14367-d947-472a-b24c-c2a1e7183e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:20.725546 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be14367-d947-472a-b24c-c2a1e7183e05 admin admin] PATH_INFO: `/v3/users/2aea4a0e0ec74c538735b890fb774b14` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:20.726695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be14367-d947-472a-b24c-c2a1e7183e05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aea4a0e0ec74c538735b890fb774b14)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:20.728729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be14367-d947-472a-b24c-c2a1e7183e05 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:20.920244 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8be14367-d947-472a-b24c-c2a1e7183e05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aea4a0e0ec74c538735b890fb774b14'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:53:20.935403 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8be14367-d947-472a-b24c-c2a1e7183e05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aea4a0e0ec74c538735b890fb774b14'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:53:21.019239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8be14367-d947-472a-b24c-c2a1e7183e05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aea4a0e0ec74c538735b890fb774b14'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:53:21.019874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8be14367-d947-472a-b24c-c2a1e7183e05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aea4a0e0ec74c538735b890fb774b14'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:53:21.027963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8be14367-d947-472a-b24c-c2a1e7183e05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aea4a0e0ec74c538735b890fb774b14'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:53:21.034813 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1245/2502] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:53:20 2022] DELETE /identity/v3/users/2aea4a0e0ec74c538735b890fb774b14 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:53:21.315190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a8099-abf0-44df-a656-61670025e016 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:21.326003 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a8099-abf0-44df-a656-61670025e016 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:21.394413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6264257-7e41-4740-a955-d7bbab862173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:21.395071 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6264257-7e41-4740-a955-d7bbab862173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:21.395410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6264257-7e41-4740-a955-d7bbab862173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:21.395607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6264257-7e41-4740-a955-d7bbab862173 admin admin] PATH_INFO: `/v3/projects/bcb554b6cf504ca9a30a1ea871364e3d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:21.396777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6264257-7e41-4740-a955-d7bbab862173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb554b6cf504ca9a30a1ea871364e3d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:21.398907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6264257-7e41-4740-a955-d7bbab862173 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:21.535132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e6264257-7e41-4740-a955-d7bbab862173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb554b6cf504ca9a30a1ea871364e3d', 'request_id': 'req-e6264257-7e41-4740-a955-d7bbab862173'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:53:21.601801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e6264257-7e41-4740-a955-d7bbab862173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb554b6cf504ca9a30a1ea871364e3d', 'request_id': 'req-e6264257-7e41-4740-a955-d7bbab862173'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:53:21.608998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e6264257-7e41-4740-a955-d7bbab862173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb554b6cf504ca9a30a1ea871364e3d', 'request_id': 'req-e6264257-7e41-4740-a955-d7bbab862173'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:53:21.621456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e6264257-7e41-4740-a955-d7bbab862173 admin admin] The token cache is being invalidate because project bcb554b6cf504ca9a30a1ea871364e3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:53:21.621744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e6264257-7e41-4740-a955-d7bbab862173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:53:21.622436 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e6264257-7e41-4740-a955-d7bbab862173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:53:21.624486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1258/2503] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:53:21 2022] DELETE /identity/v3/projects/bcb554b6cf504ca9a30a1ea871364e3d => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:53:21.632387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb56e7-1c7a-49ab-987f-2ef839acab7e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:21.632677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb56e7-1c7a-49ab-987f-2ef839acab7e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:21.632928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb56e7-1c7a-49ab-987f-2ef839acab7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:21.653302 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-78cb56e7-1c7a-49ab-987f-2ef839acab7e None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:53:21.654943 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-78cb56e7-1c7a-49ab-987f-2ef839acab7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:21.723649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1246/2504] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:53:21.731860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e3d9a-6e1e-4b83-b326-82bede2b6a1a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:21.732112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e3d9a-6e1e-4b83-b326-82bede2b6a1a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:21.732371 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e3d9a-6e1e-4b83-b326-82bede2b6a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:21.755533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-393e3d9a-6e1e-4b83-b326-82bede2b6a1a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:53:21.757002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-393e3d9a-6e1e-4b83-b326-82bede2b6a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:21.766856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1259/2505] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:53:21.771426 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb56e7-1c7a-49ab-987f-2ef839acab7e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:21.782105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb56e7-1c7a-49ab-987f-2ef839acab7e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:21.788527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe1f91-6fe2-4f8a-aebe-e71be9db6fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:21.789155 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe1f91-6fe2-4f8a-aebe-e71be9db6fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:21.789432 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe1f91-6fe2-4f8a-aebe-e71be9db6fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:21.789709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe1f91-6fe2-4f8a-aebe-e71be9db6fdd admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:21.791004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe1f91-6fe2-4f8a-aebe-e71be9db6fdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:53:21.791004 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe1f91-6fe2-4f8a-aebe-e71be9db6fdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:21.792296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe1f91-6fe2-4f8a-aebe-e71be9db6fdd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:21.806087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1247/2506] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:53:21.811863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e3d9a-6e1e-4b83-b326-82bede2b6a1a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:21.827758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e3d9a-6e1e-4b83-b326-82bede2b6a1a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:21.832988 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f489c-130c-46ce-ac0a-66c39d15b42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:21.833790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f489c-130c-46ce-ac0a-66c39d15b42c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:21.834013 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f489c-130c-46ce-ac0a-66c39d15b42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:21.834234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f489c-130c-46ce-ac0a-66c39d15b42c admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:21.834890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f489c-130c-46ce-ac0a-66c39d15b42c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:21.836796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f489c-130c-46ce-ac0a-66c39d15b42c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:21.860403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:53:21.949341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1260/2507] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:53:21 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:53:21.953347 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe1f91-6fe2-4f8a-aebe-e71be9db6fdd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:21.963937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe1f91-6fe2-4f8a-aebe-e71be9db6fdd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:22.024167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20802b6d-5ee2-4039-82b7-60f6b2407304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:22.024801 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20802b6d-5ee2-4039-82b7-60f6b2407304 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:22.025077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20802b6d-5ee2-4039-82b7-60f6b2407304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:22.025345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20802b6d-5ee2-4039-82b7-60f6b2407304 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:22.025977 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20802b6d-5ee2-4039-82b7-60f6b2407304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:22.027913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20802b6d-5ee2-4039-82b7-60f6b2407304 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:22.036445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:53:22.112073 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1248/2508] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:53:21 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:53:22.116400 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f489c-130c-46ce-ac0a-66c39d15b42c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:22.126982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f489c-130c-46ce-ac0a-66c39d15b42c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:22.131706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa5304-d081-4204-b597-f63df3ee44b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:22.132327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa5304-d081-4204-b597-f63df3ee44b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:22.132584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa5304-d081-4204-b597-f63df3ee44b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:22.132853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa5304-d081-4204-b597-f63df3ee44b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:22.133492 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa5304-d081-4204-b597-f63df3ee44b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:22.135182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa5304-d081-4204-b597-f63df3ee44b7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:22.145745 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1261/2509] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:53:22.149939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20802b6d-5ee2-4039-82b7-60f6b2407304 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:22.160894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20802b6d-5ee2-4039-82b7-60f6b2407304 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:22.166707 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ef0de-3717-4225-a15b-220815b2a7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:22.171425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ef0de-3717-4225-a15b-220815b2a7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:22.171425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ef0de-3717-4225-a15b-220815b2a7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:22.171425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ef0de-3717-4225-a15b-220815b2a7a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:22.171425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ef0de-3717-4225-a15b-220815b2a7a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:22.171425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ef0de-3717-4225-a15b-220815b2a7a0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:22.182084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1249/2510] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:53:22.186445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa5304-d081-4204-b597-f63df3ee44b7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:22.196549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa5304-d081-4204-b597-f63df3ee44b7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:22.201298 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579acaf-ea82-4177-b5f6-34de6d700a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:22.201911 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579acaf-ea82-4177-b5f6-34de6d700a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:22.202220 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579acaf-ea82-4177-b5f6-34de6d700a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:22.202496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579acaf-ea82-4177-b5f6-34de6d700a08 admin admin] PATH_INFO: `/v3/projects/c0401850bdf44aaaa6c37ed752708f53/users/c1c1de0489104713a90c59b89a284321/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:22.218071 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579acaf-ea82-4177-b5f6-34de6d700a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0401850bdf44aaaa6c37ed752708f53, user_id=c1c1de0489104713a90c59b89a284321, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:22.220769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579acaf-ea82-4177-b5f6-34de6d700a08 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:22.291450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1262/2511] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:53:22 2022] PUT /identity/v3/projects/c0401850bdf44aaaa6c37ed752708f53/users/c1c1de0489104713a90c59b89a284321/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:53:22.297031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e993c8-b46c-4aa6-b74a-61a7b9769c2a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:22.297397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e993c8-b46c-4aa6-b74a-61a7b9769c2a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:22.297841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e993c8-b46c-4aa6-b74a-61a7b9769c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:22.318009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-49e993c8-b46c-4aa6-b74a-61a7b9769c2a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:53:22.319260 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-49e993c8-b46c-4aa6-b74a-61a7b9769c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:22.384445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1250/2512] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:53:22.396918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579acaf-ea82-4177-b5f6-34de6d700a08 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:22.396918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5579acaf-ea82-4177-b5f6-34de6d700a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:22.471139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579acaf-ea82-4177-b5f6-34de6d700a08 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:22.474917 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270f216-36fb-47d3-8bf8-e77798c245fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:22.475530 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270f216-36fb-47d3-8bf8-e77798c245fc service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:22.475810 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270f216-36fb-47d3-8bf8-e77798c245fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:22.476077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270f216-36fb-47d3-8bf8-e77798c245fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:22.486090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270f216-36fb-47d3-8bf8-e77798c245fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:22.487559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270f216-36fb-47d3-8bf8-e77798c245fc service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:22.497730 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1263/2513] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:53:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:53:28.278334 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18309-70d6-44bd-aa58-e83b46649047 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:28.278334 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18309-70d6-44bd-aa58-e83b46649047 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:28.278897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18309-70d6-44bd-aa58-e83b46649047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:28.311344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-05a18309-70d6-44bd-aa58-e83b46649047 None None] MFA Rules not processed for user `c1c1de0489104713a90c59b89a284321`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:53:28.312601 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-05a18309-70d6-44bd-aa58-e83b46649047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:28.385602 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1251/2514] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:53:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:53:28.400599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270f216-36fb-47d3-8bf8-e77798c245fc service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:28.405514 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9270f216-36fb-47d3-8bf8-e77798c245fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:28.495278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270f216-36fb-47d3-8bf8-e77798c245fc service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:28.501369 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc7e44-14b2-4a55-8fab-1ac1f0c65187 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:28.502007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc7e44-14b2-4a55-8fab-1ac1f0c65187 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:28.502320 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc7e44-14b2-4a55-8fab-1ac1f0c65187 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:28.502760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc7e44-14b2-4a55-8fab-1ac1f0c65187 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:28.515287 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccc7e44-14b2-4a55-8fab-1ac1f0c65187 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:28.517128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccc7e44-14b2-4a55-8fab-1ac1f0c65187 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:28.527485 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1264/2515] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:53:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:53:31.119563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88009216-44e4-488e-a250-44015c22c5eb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:31.120045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88009216-44e4-488e-a250-44015c22c5eb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:31.120169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88009216-44e4-488e-a250-44015c22c5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:31.147748 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-88009216-44e4-488e-a250-44015c22c5eb None None] MFA Rules not processed for user `c1c1de0489104713a90c59b89a284321`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:53:31.148283 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88009216-44e4-488e-a250-44015c22c5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:31.174492 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1252/2516] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:53:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:53:34.115421 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3f5f4-46df-4ac7-ad31-45d20184fb5d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:34.116354 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3f5f4-46df-4ac7-ad31-45d20184fb5d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:34.116354 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3f5f4-46df-4ac7-ad31-45d20184fb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:34.148236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-85f3f5f4-46df-4ac7-ad31-45d20184fb5d None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:53:34.149711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-85f3f5f4-46df-4ac7-ad31-45d20184fb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:34.203688 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:53:34.216654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1265/2517] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:53:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:53:34.375389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88009216-44e4-488e-a250-44015c22c5eb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:34.376824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88009216-44e4-488e-a250-44015c22c5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:34.479123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:53:34.483946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88009216-44e4-488e-a250-44015c22c5eb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:34.489630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eba74-942f-4fa7-b01e-662907b62090 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:34.492480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eba74-942f-4fa7-b01e-662907b62090 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:34.492480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eba74-942f-4fa7-b01e-662907b62090 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:34.492480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eba74-942f-4fa7-b01e-662907b62090 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:34.499974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733eba74-942f-4fa7-b01e-662907b62090 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:34.501852 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733eba74-942f-4fa7-b01e-662907b62090 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:34.524768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1253/2518] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:53:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:53:44.271888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:53:44.545333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:53:58.330701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448c67e-dfe6-44ad-bd35-b16f5fa63cac None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:58.331531 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448c67e-dfe6-44ad-bd35-b16f5fa63cac None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:58.331696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448c67e-dfe6-44ad-bd35-b16f5fa63cac None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:58.333882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1266/2519] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:53:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:53:58.346322 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2bd9d-e7c4-4147-ae32-2fe323e571d5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:58.348064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2bd9d-e7c4-4147-ae32-2fe323e571d5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:58.350972 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2bd9d-e7c4-4147-ae32-2fe323e571d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:58.435268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d0a2bd9d-e7c4-4147-ae32-2fe323e571d5 None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:53:58.442190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d0a2bd9d-e7c4-4147-ae32-2fe323e571d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:58.466898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1254/2520] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:53:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:53:58.477505 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448c67e-dfe6-44ad-bd35-b16f5fa63cac None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:58.480447 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0448c67e-dfe6-44ad-bd35-b16f5fa63cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:58.631937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448c67e-dfe6-44ad-bd35-b16f5fa63cac None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:58.638601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536d92c-6c0d-47a4-8361-b49f93ce6b82 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b86978df61245ae88fead154193fcd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:58.639802 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536d92c-6c0d-47a4-8361-b49f93ce6b82 service glance] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:58.640421 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536d92c-6c0d-47a4-8361-b49f93ce6b82 service glance] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:58.641046 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536d92c-6c0d-47a4-8361-b49f93ce6b82 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:58.662545 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536d92c-6c0d-47a4-8361-b49f93ce6b82 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:58.666528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536d92c-6c0d-47a4-8361-b49f93ce6b82 service glance] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:58.689548 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1267/2521] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:53:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:53:58.939665 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4620e-1560-4114-8449-bf92d678df4b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:58.940000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4620e-1560-4114-8449-bf92d678df4b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:58.940360 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4620e-1560-4114-8449-bf92d678df4b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:58.942134 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1255/2522] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:53:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:53:58.948136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1b3fa-6feb-48a8-a9d1-db2133178199 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:58.948540 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1b3fa-6feb-48a8-a9d1-db2133178199 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:58.948838 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1b3fa-6feb-48a8-a9d1-db2133178199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:58.972804 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-75b1b3fa-6feb-48a8-a9d1-db2133178199 None None] MFA Rules not processed for user `92867e15dcf649adad78bf7a29a996f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:53:58.974739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-75b1b3fa-6feb-48a8-a9d1-db2133178199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:58.981152 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1268/2523] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:53:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:53:59.000209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4620e-1560-4114-8449-bf92d678df4b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:59.002102 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eaf4620e-1560-4114-8449-bf92d678df4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:59.019313 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4620e-1560-4114-8449-bf92d678df4b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:59.024321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd650c4-5326-4837-a76a-19985c862701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:59.024991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd650c4-5326-4837-a76a-19985c862701 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:59.025276 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd650c4-5326-4837-a76a-19985c862701 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:59.025531 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd650c4-5326-4837-a76a-19985c862701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:59.034858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd650c4-5326-4837-a76a-19985c862701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:59.036385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd650c4-5326-4837-a76a-19985c862701 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:59.045431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1256/2524] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:53:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:53:59.152629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd5dfb-7bc1-4215-98ea-806ec0622ad5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:59.154840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd5dfb-7bc1-4215-98ea-806ec0622ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:59.155434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd5dfb-7bc1-4215-98ea-806ec0622ad5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:59.159431 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1269/2525] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 08:53:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 08:53:59.165515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd608e-e60e-42a0-a432-14e53afa8465 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:59.165860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd608e-e60e-42a0-a432-14e53afa8465 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:59.166190 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd608e-e60e-42a0-a432-14e53afa8465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:59.196758 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-94bd608e-e60e-42a0-a432-14e53afa8465 None None] MFA Rules not processed for user `58c8502b91ea4ba79e85818727422410`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:53:59.198315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-94bd608e-e60e-42a0-a432-14e53afa8465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:53:59.221635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1257/2526] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 08:53:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:53:59.239710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd5dfb-7bc1-4215-98ea-806ec0622ad5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:53:59.267743 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd5dfb-7bc1-4215-98ea-806ec0622ad5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:53:59.277657 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7e9c9-d188-491a-9fcc-c0c26b8b9ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:53:59.279332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7e9c9-d188-491a-9fcc-c0c26b8b9ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:53:59.279775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7e9c9-d188-491a-9fcc-c0c26b8b9ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:53:59.280194 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7e9c9-d188-491a-9fcc-c0c26b8b9ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:53:59.301912 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7e9c9-d188-491a-9fcc-c0c26b8b9ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:53:59.303813 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7e9c9-d188-491a-9fcc-c0c26b8b9ab4 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:53:59.325127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1270/2527] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:53:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:54:07.665056 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2131f-6349-41e9-b913-2bf2def4c5ee None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:07.665669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2131f-6349-41e9-b913-2bf2def4c5ee None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:07.665923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2131f-6349-41e9-b913-2bf2def4c5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:07.699617 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a0f2131f-6349-41e9-b913-2bf2def4c5ee None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:07.701955 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a0f2131f-6349-41e9-b913-2bf2def4c5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:07.713568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1258/2528] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 08:54:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:54:07.734127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7e9c9-d188-491a-9fcc-c0c26b8b9ab4 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:07.762202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7e9c9-d188-491a-9fcc-c0c26b8b9ab4 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:07.770002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126d4f7-b944-4046-b7ef-7b0500743c34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:07.771612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126d4f7-b944-4046-b7ef-7b0500743c34 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:07.771888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126d4f7-b944-4046-b7ef-7b0500743c34 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:07.772359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126d4f7-b944-4046-b7ef-7b0500743c34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:07.783706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2126d4f7-b944-4046-b7ef-7b0500743c34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:07.785341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2126d4f7-b944-4046-b7ef-7b0500743c34 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:07.799216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1271/2529] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:54:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 08:54:17.690001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=15 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:17.792840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=41 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:17.957919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2131f-6349-41e9-b913-2bf2def4c5ee None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:17.959461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a0f2131f-6349-41e9-b913-2bf2def4c5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:18.033524 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2131f-6349-41e9-b913-2bf2def4c5ee None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:18.038600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0db17-25bc-4315-92af-a04dc68db847 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d03db170fb6460794e32669456bcdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:18.039231 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0db17-25bc-4315-92af-a04dc68db847 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:18.039529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0db17-25bc-4315-92af-a04dc68db847 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:18.039797 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0db17-25bc-4315-92af-a04dc68db847 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:18.041557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c4c0db17-25bc-4315-92af-a04dc68db847 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:18.054624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0db17-25bc-4315-92af-a04dc68db847 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:18.057392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0db17-25bc-4315-92af-a04dc68db847 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:18.065126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126d4f7-b944-4046-b7ef-7b0500743c34 service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:18.066956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2126d4f7-b944-4046-b7ef-7b0500743c34 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:18.074300 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1259/2530] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:54:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:54:18.085208 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126d4f7-b944-4046-b7ef-7b0500743c34 service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:18.090410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bc8cc-11c3-4329-8ed8-565b4345e498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:18.091125 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bc8cc-11c3-4329-8ed8-565b4345e498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:18.091401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bc8cc-11c3-4329-8ed8-565b4345e498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:18.091667 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bc8cc-11c3-4329-8ed8-565b4345e498 admin admin] PATH_INFO: `/v3/users/bae09cfbf3844b1682d8924e7d9578ee` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:18.092822 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bc8cc-11c3-4329-8ed8-565b4345e498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bae09cfbf3844b1682d8924e7d9578ee)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:18.095094 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bc8cc-11c3-4329-8ed8-565b4345e498 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:18.273554 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7f8bc8cc-11c3-4329-8ed8-565b4345e498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bae09cfbf3844b1682d8924e7d9578ee'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:18.274573 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7f8bc8cc-11c3-4329-8ed8-565b4345e498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bae09cfbf3844b1682d8924e7d9578ee'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:18.368147 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7f8bc8cc-11c3-4329-8ed8-565b4345e498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bae09cfbf3844b1682d8924e7d9578ee'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:18.385525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7f8bc8cc-11c3-4329-8ed8-565b4345e498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bae09cfbf3844b1682d8924e7d9578ee'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:18.400862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7f8bc8cc-11c3-4329-8ed8-565b4345e498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bae09cfbf3844b1682d8924e7d9578ee'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:18.408736 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1272/2531] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:54:18 2022] DELETE /identity/v3/users/bae09cfbf3844b1682d8924e7d9578ee => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:54:18.792735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0db17-25bc-4315-92af-a04dc68db847 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:18.805240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0db17-25bc-4315-92af-a04dc68db847 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:18.873408 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:18.874098 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:18.879580 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:18.879580 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] PATH_INFO: `/v3/projects/94ceae42bd11404eaea28d523b98126e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:18.879580 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94ceae42bd11404eaea28d523b98126e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:18.879580 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:19.005778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94ceae42bd11404eaea28d523b98126e', 'request_id': 'req-668f87ba-992e-4c4d-848e-8a23fbe465f3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:19.082968 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94ceae42bd11404eaea28d523b98126e', 'request_id': 'req-668f87ba-992e-4c4d-848e-8a23fbe465f3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:19.090101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94ceae42bd11404eaea28d523b98126e', 'request_id': 'req-668f87ba-992e-4c4d-848e-8a23fbe465f3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:19.103183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] The token cache is being invalidate because project 94ceae42bd11404eaea28d523b98126e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:54:19.103529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:19.104171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:19.106231 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1260/2532] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:54:18 2022] DELETE /identity/v3/projects/94ceae42bd11404eaea28d523b98126e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:24.817364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b43bc-2283-44ee-8a94-7aacb6f1bf82 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:24.817865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b43bc-2283-44ee-8a94-7aacb6f1bf82 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:24.817980 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b43bc-2283-44ee-8a94-7aacb6f1bf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:24.839582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-880b43bc-2283-44ee-8a94-7aacb6f1bf82 None None] MFA Rules not processed for user `dab0506091034403bf2f0fe5f7538cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:24.840924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-880b43bc-2283-44ee-8a94-7aacb6f1bf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:24.906412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1273/2533] 10.222.0.44 () {58 vars in 967 bytes} [Sun Sep 11 08:54:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:24.919820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:24.921266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:24.944170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:24.993203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f87ba-992e-4c4d-848e-8a23fbe465f3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:24.999183 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12637f88-75b1-4bf3-9850-093ca5091997 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e60a2198022c4f979a6e5e9057558c33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:24.999789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12637f88-75b1-4bf3-9850-093ca5091997 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:25.000070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12637f88-75b1-4bf3-9850-093ca5091997 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:25.000342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12637f88-75b1-4bf3-9850-093ca5091997 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:25.008304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12637f88-75b1-4bf3-9850-093ca5091997 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:25.013333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12637f88-75b1-4bf3-9850-093ca5091997 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:25.019276 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1261/2534] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 08:54:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 08:54:25.122749 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b43bc-2283-44ee-8a94-7aacb6f1bf82 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:25.124210 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-880b43bc-2283-44ee-8a94-7aacb6f1bf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:25.187100 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b43bc-2283-44ee-8a94-7aacb6f1bf82 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:25.190572 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89561980-cb8c-426e-8ebb-733e3c0d422b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:25.191171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89561980-cb8c-426e-8ebb-733e3c0d422b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:25.191443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89561980-cb8c-426e-8ebb-733e3c0d422b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:25.191706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89561980-cb8c-426e-8ebb-733e3c0d422b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:25.193384 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-89561980-cb8c-426e-8ebb-733e3c0d422b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:25.224613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:25.260677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89561980-cb8c-426e-8ebb-733e3c0d422b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:25.262131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89561980-cb8c-426e-8ebb-733e3c0d422b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:25.274295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1274/2535] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:54:29.957229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12637f88-75b1-4bf3-9850-093ca5091997 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:29.958785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-12637f88-75b1-4bf3-9850-093ca5091997 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:29.974803 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12637f88-75b1-4bf3-9850-093ca5091997 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:29.980249 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ebc83-8c28-4747-acda-79768cd91339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:29.980877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ebc83-8c28-4747-acda-79768cd91339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:29.981187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ebc83-8c28-4747-acda-79768cd91339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:29.982013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ebc83-8c28-4747-acda-79768cd91339 admin admin] PATH_INFO: `/v3/users/3abba619decf448db477533c663004bb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:29.983643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ebc83-8c28-4747-acda-79768cd91339 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3abba619decf448db477533c663004bb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:29.985888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ebc83-8c28-4747-acda-79768cd91339 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:30.249619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f7ebc83-8c28-4747-acda-79768cd91339 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3abba619decf448db477533c663004bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:30.268399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f7ebc83-8c28-4747-acda-79768cd91339 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3abba619decf448db477533c663004bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:30.383763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f7ebc83-8c28-4747-acda-79768cd91339 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3abba619decf448db477533c663004bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:30.384390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f7ebc83-8c28-4747-acda-79768cd91339 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3abba619decf448db477533c663004bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:30.393199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f7ebc83-8c28-4747-acda-79768cd91339 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3abba619decf448db477533c663004bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:30.399728 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1262/2536] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:54:29 2022] DELETE /identity/v3/users/3abba619decf448db477533c663004bb => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:30.708251 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89561980-cb8c-426e-8ebb-733e3c0d422b service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:30.719768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89561980-cb8c-426e-8ebb-733e3c0d422b service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:30.786707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15bb7a-4018-4d2f-9ba5-df2100f52994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:30.787356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15bb7a-4018-4d2f-9ba5-df2100f52994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:30.787704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15bb7a-4018-4d2f-9ba5-df2100f52994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:30.788044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15bb7a-4018-4d2f-9ba5-df2100f52994 admin admin] PATH_INFO: `/v3/projects/347504c603af4937b829f24349b013c2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:30.789259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15bb7a-4018-4d2f-9ba5-df2100f52994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=347504c603af4937b829f24349b013c2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:30.791372 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15bb7a-4018-4d2f-9ba5-df2100f52994 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:30.911315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9e15bb7a-4018-4d2f-9ba5-df2100f52994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347504c603af4937b829f24349b013c2', 'request_id': 'req-9e15bb7a-4018-4d2f-9ba5-df2100f52994'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:30.995690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9e15bb7a-4018-4d2f-9ba5-df2100f52994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347504c603af4937b829f24349b013c2', 'request_id': 'req-9e15bb7a-4018-4d2f-9ba5-df2100f52994'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:31.004525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9e15bb7a-4018-4d2f-9ba5-df2100f52994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347504c603af4937b829f24349b013c2', 'request_id': 'req-9e15bb7a-4018-4d2f-9ba5-df2100f52994'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:31.018135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9e15bb7a-4018-4d2f-9ba5-df2100f52994 admin admin] The token cache is being invalidate because project 347504c603af4937b829f24349b013c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:54:31.018448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9e15bb7a-4018-4d2f-9ba5-df2100f52994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:31.019242 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9e15bb7a-4018-4d2f-9ba5-df2100f52994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:31.020831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1275/2537] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:54:30 2022] DELETE /identity/v3/projects/347504c603af4937b829f24349b013c2 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:31.025784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966abce-8794-4c9a-8568-7b413505eaff None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:31.026332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966abce-8794-4c9a-8568-7b413505eaff None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:31.026332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966abce-8794-4c9a-8568-7b413505eaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:31.046547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1966abce-8794-4c9a-8568-7b413505eaff None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:31.047756 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1966abce-8794-4c9a-8568-7b413505eaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:31.113662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1263/2538] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:31.122475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809f261-f157-4c61-8d1e-a7439594ee84 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:31.122755 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809f261-f157-4c61-8d1e-a7439594ee84 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:31.123019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809f261-f157-4c61-8d1e-a7439594ee84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:31.143651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9809f261-f157-4c61-8d1e-a7439594ee84 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:31.144853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9809f261-f157-4c61-8d1e-a7439594ee84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:31.161789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1276/2539] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:31.166367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966abce-8794-4c9a-8568-7b413505eaff None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:31.176771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966abce-8794-4c9a-8568-7b413505eaff None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:31.182298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def144a0-bdbc-42d8-81e4-b2e3de8b0104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:31.182910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def144a0-bdbc-42d8-81e4-b2e3de8b0104 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:31.183188 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def144a0-bdbc-42d8-81e4-b2e3de8b0104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:31.183461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def144a0-bdbc-42d8-81e4-b2e3de8b0104 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:31.184286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def144a0-bdbc-42d8-81e4-b2e3de8b0104 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:54:31.184835 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def144a0-bdbc-42d8-81e4-b2e3de8b0104 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:31.187396 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def144a0-bdbc-42d8-81e4-b2e3de8b0104 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:31.200614 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1264/2540] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:54:31.205010 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809f261-f157-4c61-8d1e-a7439594ee84 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:31.214807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809f261-f157-4c61-8d1e-a7439594ee84 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:31.221228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b615d-511b-4aad-ad2d-6f66ada0845f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:31.222848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b615d-511b-4aad-ad2d-6f66ada0845f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:31.223405 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b615d-511b-4aad-ad2d-6f66ada0845f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:31.223893 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b615d-511b-4aad-ad2d-6f66ada0845f admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:31.224794 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b615d-511b-4aad-ad2d-6f66ada0845f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:31.228084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b615d-511b-4aad-ad2d-6f66ada0845f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:31.323323 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1277/2541] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:54:31 2022] POST /identity/v3/projects => generated 361 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:31.327914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def144a0-bdbc-42d8-81e4-b2e3de8b0104 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:31.337994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def144a0-bdbc-42d8-81e4-b2e3de8b0104 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:31.343430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:31.397626 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38709cf-67c0-409e-89e0-846546394b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:31.398275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38709cf-67c0-409e-89e0-846546394b6a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:31.398559 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38709cf-67c0-409e-89e0-846546394b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:31.398820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38709cf-67c0-409e-89e0-846546394b6a admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:31.399506 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38709cf-67c0-409e-89e0-846546394b6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:31.401415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38709cf-67c0-409e-89e0-846546394b6a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:31.545337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1265/2542] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:54:31 2022] POST /identity/v3/users => generated 281 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:31.550863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b615d-511b-4aad-ad2d-6f66ada0845f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:31.561525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b615d-511b-4aad-ad2d-6f66ada0845f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:31.566017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac980696-7e95-4019-b070-f5e58d18fe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:31.566722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac980696-7e95-4019-b070-f5e58d18fe67 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:31.567012 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac980696-7e95-4019-b070-f5e58d18fe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:31.567419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac980696-7e95-4019-b070-f5e58d18fe67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:31.568291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac980696-7e95-4019-b070-f5e58d18fe67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:31.570408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac980696-7e95-4019-b070-f5e58d18fe67 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:31.583089 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1278/2543] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:54:31.587086 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38709cf-67c0-409e-89e0-846546394b6a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:31.597700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38709cf-67c0-409e-89e0-846546394b6a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:31.602279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61204c72-1f73-45c0-837e-9a5624fd5417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:31.602864 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61204c72-1f73-45c0-837e-9a5624fd5417 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:31.603139 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61204c72-1f73-45c0-837e-9a5624fd5417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:31.603407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61204c72-1f73-45c0-837e-9a5624fd5417 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:31.604036 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61204c72-1f73-45c0-837e-9a5624fd5417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:31.605604 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61204c72-1f73-45c0-837e-9a5624fd5417 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:31.617259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1266/2544] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:54:31.621482 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac980696-7e95-4019-b070-f5e58d18fe67 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:31.632640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac980696-7e95-4019-b070-f5e58d18fe67 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:31.638313 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34613310-cb6b-4107-ad16-a9c360b954fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:31.638987 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34613310-cb6b-4107-ad16-a9c360b954fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:31.639257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34613310-cb6b-4107-ad16-a9c360b954fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:31.639524 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34613310-cb6b-4107-ad16-a9c360b954fd admin admin] PATH_INFO: `/v3/projects/1840ee00cfce40c2b096068726de47a0/users/e938a2704041438bbddf2382da5db86d/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:31.655423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34613310-cb6b-4107-ad16-a9c360b954fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1840ee00cfce40c2b096068726de47a0, user_id=e938a2704041438bbddf2382da5db86d, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:31.658057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34613310-cb6b-4107-ad16-a9c360b954fd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:31.660547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:31.745692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1279/2545] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:54:31 2022] PUT /identity/v3/projects/1840ee00cfce40c2b096068726de47a0/users/e938a2704041438bbddf2382da5db86d/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:31.756197 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29c20b-6059-467b-a6a5-094cb4c2d287 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:31.756476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29c20b-6059-467b-a6a5-094cb4c2d287 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:31.756775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29c20b-6059-467b-a6a5-094cb4c2d287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:31.787399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5d29c20b-6059-467b-a6a5-094cb4c2d287 None None] MFA Rules not processed for user `e938a2704041438bbddf2382da5db86d`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:31.788721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5d29c20b-6059-467b-a6a5-094cb4c2d287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:31.857203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1267/2546] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:54:31.862000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34613310-cb6b-4107-ad16-a9c360b954fd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:31.937814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34613310-cb6b-4107-ad16-a9c360b954fd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:31.943136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529bd9c-dc0d-47c4-be40-66b34781dc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:31.943814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529bd9c-dc0d-47c4-be40-66b34781dc1c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:31.944191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529bd9c-dc0d-47c4-be40-66b34781dc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:31.944527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529bd9c-dc0d-47c4-be40-66b34781dc1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:31.945245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529bd9c-dc0d-47c4-be40-66b34781dc1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:31.947303 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529bd9c-dc0d-47c4-be40-66b34781dc1c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:32.055395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1280/2547] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:54:31 2022] POST /identity/v3/projects => generated 365 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:32.059729 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29c20b-6059-467b-a6a5-094cb4c2d287 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:32.071224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29c20b-6059-467b-a6a5-094cb4c2d287 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:32.133710 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31d29e-3557-4140-a0f8-64dc9f64aebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:32.134260 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31d29e-3557-4140-a0f8-64dc9f64aebb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:32.134543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31d29e-3557-4140-a0f8-64dc9f64aebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:32.134813 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31d29e-3557-4140-a0f8-64dc9f64aebb admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:32.135437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31d29e-3557-4140-a0f8-64dc9f64aebb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:32.137897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31d29e-3557-4140-a0f8-64dc9f64aebb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:32.213904 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1268/2548] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:54:32 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:32.222933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529bd9c-dc0d-47c4-be40-66b34781dc1c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:32.235718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529bd9c-dc0d-47c4-be40-66b34781dc1c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:32.240631 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c56e9f-3002-4f22-9e68-e135ae91a160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:32.241224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c56e9f-3002-4f22-9e68-e135ae91a160 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:32.241496 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c56e9f-3002-4f22-9e68-e135ae91a160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:32.241761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c56e9f-3002-4f22-9e68-e135ae91a160 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:32.242426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c56e9f-3002-4f22-9e68-e135ae91a160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:32.244228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c56e9f-3002-4f22-9e68-e135ae91a160 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:32.256134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1281/2549] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:54:32.260501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31d29e-3557-4140-a0f8-64dc9f64aebb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:32.272400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31d29e-3557-4140-a0f8-64dc9f64aebb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:32.277131 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f6c9e-5540-4ef9-a95a-7f177ca7788f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:32.277795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f6c9e-5540-4ef9-a95a-7f177ca7788f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:32.278067 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f6c9e-5540-4ef9-a95a-7f177ca7788f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:32.278366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f6c9e-5540-4ef9-a95a-7f177ca7788f admin admin] PATH_INFO: `/v3/projects/4747a38ba1124d8aae70fee43dd9aad6/users/1a063cb1116c45459235c29539d33fcf/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:32.280613 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:32.294538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f6c9e-5540-4ef9-a95a-7f177ca7788f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4747a38ba1124d8aae70fee43dd9aad6, user_id=1a063cb1116c45459235c29539d33fcf, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:32.297553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f6c9e-5540-4ef9-a95a-7f177ca7788f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:32.431222 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1269/2550] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:54:32 2022] PUT /identity/v3/projects/4747a38ba1124d8aae70fee43dd9aad6/users/1a063cb1116c45459235c29539d33fcf/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:32.435825 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c56e9f-3002-4f22-9e68-e135ae91a160 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:32.445998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c56e9f-3002-4f22-9e68-e135ae91a160 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:32.506530 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdda23e-6403-47d4-a7e4-529a1b734ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:32.507140 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdda23e-6403-47d4-a7e4-529a1b734ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:32.507413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdda23e-6403-47d4-a7e4-529a1b734ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:32.507680 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdda23e-6403-47d4-a7e4-529a1b734ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:32.508315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdda23e-6403-47d4-a7e4-529a1b734ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:32.509907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdda23e-6403-47d4-a7e4-529a1b734ac6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:32.520211 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1282/2551] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:54:32.525074 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f6c9e-5540-4ef9-a95a-7f177ca7788f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:32.535657 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f6c9e-5540-4ef9-a95a-7f177ca7788f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:32.540398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ee6f8-93b2-490d-8177-c657a94023dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:32.541049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ee6f8-93b2-490d-8177-c657a94023dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:32.541327 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ee6f8-93b2-490d-8177-c657a94023dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:32.541595 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ee6f8-93b2-490d-8177-c657a94023dd admin admin] PATH_INFO: `/v3/projects/4747a38ba1124d8aae70fee43dd9aad6/users/1a063cb1116c45459235c29539d33fcf/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:32.544253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ee6f8-93b2-490d-8177-c657a94023dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4747a38ba1124d8aae70fee43dd9aad6, user_id=1a063cb1116c45459235c29539d33fcf, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:32.546945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ee6f8-93b2-490d-8177-c657a94023dd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:32.558048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1270/2552] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:54:32 2022] PUT /identity/v3/projects/4747a38ba1124d8aae70fee43dd9aad6/users/1a063cb1116c45459235c29539d33fcf/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:32.568966 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66862240-82fd-4161-9954-92137f167633 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:32.569246 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66862240-82fd-4161-9954-92137f167633 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:32.569515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66862240-82fd-4161-9954-92137f167633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:32.606876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-66862240-82fd-4161-9954-92137f167633 None None] MFA Rules not processed for user `1a063cb1116c45459235c29539d33fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:32.608083 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-66862240-82fd-4161-9954-92137f167633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:32.660584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:32.680073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1283/2553] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:32.691826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ee6f8-93b2-490d-8177-c657a94023dd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:32.693516 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6c3ee6f8-93b2-490d-8177-c657a94023dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:32.758558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ee6f8-93b2-490d-8177-c657a94023dd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:32.762495 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc9708-cbfd-417e-aedb-d4f4fad98687 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:32.763108 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc9708-cbfd-417e-aedb-d4f4fad98687 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:32.763378 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc9708-cbfd-417e-aedb-d4f4fad98687 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:32.764145 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc9708-cbfd-417e-aedb-d4f4fad98687 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:32.774635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc9708-cbfd-417e-aedb-d4f4fad98687 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:32.776111 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc9708-cbfd-417e-aedb-d4f4fad98687 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:32.786741 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1271/2554] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:54:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:54:33.028944 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66862240-82fd-4161-9954-92137f167633 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:33.039580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66862240-82fd-4161-9954-92137f167633 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:33.043150 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e6578-9a16-4fe0-a40a-76e42c180b0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:33.043746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e6578-9a16-4fe0-a40a-76e42c180b0f service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:33.044026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e6578-9a16-4fe0-a40a-76e42c180b0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:33.044295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e6578-9a16-4fe0-a40a-76e42c180b0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:33.105108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e6578-9a16-4fe0-a40a-76e42c180b0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:33.106589 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e6578-9a16-4fe0-a40a-76e42c180b0f service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:33.116491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1284/2555] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:54:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:54:36.939272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc9708-cbfd-417e-aedb-d4f4fad98687 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:37.004890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc9708-cbfd-417e-aedb-d4f4fad98687 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:37.009826 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:37.010583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:37.010890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:37.011030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] PATH_INFO: `/v3/users/e938a2704041438bbddf2382da5db86d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:37.012159 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e938a2704041438bbddf2382da5db86d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:37.014075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:37.228587 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e938a2704041438bbddf2382da5db86d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:37.240834 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e938a2704041438bbddf2382da5db86d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:37.343454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e938a2704041438bbddf2382da5db86d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:37.344071 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e938a2704041438bbddf2382da5db86d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:37.346631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:37.352716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e938a2704041438bbddf2382da5db86d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:37.359203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1272/2556] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:54:36 2022] DELETE /identity/v3/users/e938a2704041438bbddf2382da5db86d => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:37.364064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ace0e-aa35-40cd-9196-3437f6fbfbb4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:37.364546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ace0e-aa35-40cd-9196-3437f6fbfbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:37.364663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ace0e-aa35-40cd-9196-3437f6fbfbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:37.384733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-584ace0e-aa35-40cd-9196-3437f6fbfbb4 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:37.385926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-584ace0e-aa35-40cd-9196-3437f6fbfbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:37.461475 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1285/2557] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:37.471374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:37.472946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:37.532766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349395e-8c9d-45e8-8197-6bab4f495e7c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:37.536793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad8e8d-de3e-46ec-b750-ee07fe2c7c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:37.537441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad8e8d-de3e-46ec-b750-ee07fe2c7c88 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:37.537721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad8e8d-de3e-46ec-b750-ee07fe2c7c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:37.538018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad8e8d-de3e-46ec-b750-ee07fe2c7c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:37.548253 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad8e8d-de3e-46ec-b750-ee07fe2c7c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:37.549903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad8e8d-de3e-46ec-b750-ee07fe2c7c88 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:37.562005 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1273/2558] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:54:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:54:38.038810 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ace0e-aa35-40cd-9196-3437f6fbfbb4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:38.040513 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-584ace0e-aa35-40cd-9196-3437f6fbfbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:38.054689 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ace0e-aa35-40cd-9196-3437f6fbfbb4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:38.065127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c8422-df62-41eb-b180-d6c2a57875a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:38.065127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c8422-df62-41eb-b180-d6c2a57875a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:38.065127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c8422-df62-41eb-b180-d6c2a57875a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:38.065430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c8422-df62-41eb-b180-d6c2a57875a2 admin admin] PATH_INFO: `/v3/projects/1840ee00cfce40c2b096068726de47a0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:38.066550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c8422-df62-41eb-b180-d6c2a57875a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1840ee00cfce40c2b096068726de47a0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:38.068600 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c8422-df62-41eb-b180-d6c2a57875a2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:38.220264 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-600c8422-df62-41eb-b180-d6c2a57875a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1840ee00cfce40c2b096068726de47a0', 'request_id': 'req-600c8422-df62-41eb-b180-d6c2a57875a2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:38.292255 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-600c8422-df62-41eb-b180-d6c2a57875a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1840ee00cfce40c2b096068726de47a0', 'request_id': 'req-600c8422-df62-41eb-b180-d6c2a57875a2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:38.299192 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-600c8422-df62-41eb-b180-d6c2a57875a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1840ee00cfce40c2b096068726de47a0', 'request_id': 'req-600c8422-df62-41eb-b180-d6c2a57875a2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:38.310863 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-600c8422-df62-41eb-b180-d6c2a57875a2 admin admin] The token cache is being invalidate because project 1840ee00cfce40c2b096068726de47a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:54:38.311142 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-600c8422-df62-41eb-b180-d6c2a57875a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:38.311789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-600c8422-df62-41eb-b180-d6c2a57875a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:38.313560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1286/2559] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:54:38 2022] DELETE /identity/v3/projects/1840ee00cfce40c2b096068726de47a0 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:38.317386 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad8e8d-de3e-46ec-b750-ee07fe2c7c88 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:38.327561 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad8e8d-de3e-46ec-b750-ee07fe2c7c88 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:38.388412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:38.389037 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:38.389820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:38.390090 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] PATH_INFO: `/v3/users/1a063cb1116c45459235c29539d33fcf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:38.391234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a063cb1116c45459235c29539d33fcf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:38.393202 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:38.634424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a063cb1116c45459235c29539d33fcf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:38.647626 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a063cb1116c45459235c29539d33fcf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:38.761219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a063cb1116c45459235c29539d33fcf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:38.761867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a063cb1116c45459235c29539d33fcf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:38.770478 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a063cb1116c45459235c29539d33fcf'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:38.776645 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1274/2560] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:54:38 2022] DELETE /identity/v3/users/1a063cb1116c45459235c29539d33fcf => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:39.530517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c4661-a234-431e-a959-a01bb5d41a70 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:39.531037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c4661-a234-431e-a959-a01bb5d41a70 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:39.531037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c4661-a234-431e-a959-a01bb5d41a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:39.549851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-dc6c4661-a234-431e-a959-a01bb5d41a70 None None] MFA Rules not processed for user `c1c1de0489104713a90c59b89a284321`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:39.551086 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dc6c4661-a234-431e-a959-a01bb5d41a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:39.556460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:39.607752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1287/2561] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:39.622946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:39.627169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:39.651498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:39.702996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1830e-9a30-45e5-9b0d-7ba7c5027d6c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:39.707701 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de010955-94b6-496b-a122-91212d234796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:39.708382 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de010955-94b6-496b-a122-91212d234796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:39.708702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de010955-94b6-496b-a122-91212d234796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:39.709062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de010955-94b6-496b-a122-91212d234796 admin admin] PATH_INFO: `/v3/projects/4747a38ba1124d8aae70fee43dd9aad6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:39.710295 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de010955-94b6-496b-a122-91212d234796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4747a38ba1124d8aae70fee43dd9aad6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:39.712210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de010955-94b6-496b-a122-91212d234796 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:39.879103 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-de010955-94b6-496b-a122-91212d234796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4747a38ba1124d8aae70fee43dd9aad6', 'request_id': 'req-de010955-94b6-496b-a122-91212d234796'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:39.968272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-de010955-94b6-496b-a122-91212d234796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4747a38ba1124d8aae70fee43dd9aad6', 'request_id': 'req-de010955-94b6-496b-a122-91212d234796'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:39.974365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-de010955-94b6-496b-a122-91212d234796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4747a38ba1124d8aae70fee43dd9aad6', 'request_id': 'req-de010955-94b6-496b-a122-91212d234796'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:39.986425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-de010955-94b6-496b-a122-91212d234796 admin admin] The token cache is being invalidate because project 4747a38ba1124d8aae70fee43dd9aad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:54:39.986713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-de010955-94b6-496b-a122-91212d234796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:39.987392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-de010955-94b6-496b-a122-91212d234796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:39.988933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1275/2562] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:54:39 2022] DELETE /identity/v3/projects/4747a38ba1124d8aae70fee43dd9aad6 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:39.993751 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082299be-0880-4388-9a62-358bee8e612f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:39.994035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082299be-0880-4388-9a62-358bee8e612f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:39.994334 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082299be-0880-4388-9a62-358bee8e612f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:40.014937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-082299be-0880-4388-9a62-358bee8e612f None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:40.016183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-082299be-0880-4388-9a62-358bee8e612f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:40.080004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1288/2563] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:40.089104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fd447-9368-4100-bddf-4078abb2f3a7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:40.089461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fd447-9368-4100-bddf-4078abb2f3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:40.089764 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fd447-9368-4100-bddf-4078abb2f3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:40.110885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-db2fd447-9368-4100-bddf-4078abb2f3a7 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:40.112106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-db2fd447-9368-4100-bddf-4078abb2f3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:40.120929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1276/2564] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:40.127228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082299be-0880-4388-9a62-358bee8e612f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:40.142930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082299be-0880-4388-9a62-358bee8e612f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:40.149414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72e679-45b3-4590-b2ed-379901a141ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:40.150343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72e679-45b3-4590-b2ed-379901a141ab admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:40.150716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72e679-45b3-4590-b2ed-379901a141ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:40.151095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72e679-45b3-4590-b2ed-379901a141ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:40.153286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72e679-45b3-4590-b2ed-379901a141ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:54:40.153831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72e679-45b3-4590-b2ed-379901a141ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:40.156932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea72e679-45b3-4590-b2ed-379901a141ab admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:40.174188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1289/2565] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:54:40.178735 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2fd447-9368-4100-bddf-4078abb2f3a7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:40.189115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2fd447-9368-4100-bddf-4078abb2f3a7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:40.195059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ce55e-ecb1-44a7-af19-7a1fb5fbf25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:40.195822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ce55e-ecb1-44a7-af19-7a1fb5fbf25d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:40.196093 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ce55e-ecb1-44a7-af19-7a1fb5fbf25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:40.196373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ce55e-ecb1-44a7-af19-7a1fb5fbf25d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:40.197002 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ce55e-ecb1-44a7-af19-7a1fb5fbf25d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:40.200047 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ce55e-ecb1-44a7-af19-7a1fb5fbf25d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:40.363051 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1277/2566] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:54:40 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:40.367294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72e679-45b3-4590-b2ed-379901a141ab admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:40.377409 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea72e679-45b3-4590-b2ed-379901a141ab admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:40.446748 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e93a5-cc4c-4d2e-a05f-f8670e9f0c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:40.447725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e93a5-cc4c-4d2e-a05f-f8670e9f0c8f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:40.448075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e93a5-cc4c-4d2e-a05f-f8670e9f0c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:40.448410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e93a5-cc4c-4d2e-a05f-f8670e9f0c8f admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:40.449128 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e93a5-cc4c-4d2e-a05f-f8670e9f0c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:40.451091 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e93a5-cc4c-4d2e-a05f-f8670e9f0c8f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:40.581627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1290/2567] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:54:40 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:40.583716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ce55e-ecb1-44a7-af19-7a1fb5fbf25d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:40.594857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ce55e-ecb1-44a7-af19-7a1fb5fbf25d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:40.599606 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d07efe-09c8-470d-acca-879831596e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:40.600210 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d07efe-09c8-470d-acca-879831596e38 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:40.600496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d07efe-09c8-470d-acca-879831596e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:40.600771 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d07efe-09c8-470d-acca-879831596e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:40.601413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d07efe-09c8-470d-acca-879831596e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:40.603012 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d07efe-09c8-470d-acca-879831596e38 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:40.615505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1278/2568] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:54:40.618675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e93a5-cc4c-4d2e-a05f-f8670e9f0c8f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:40.629359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e93a5-cc4c-4d2e-a05f-f8670e9f0c8f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:40.634331 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8decf209-de04-48fa-9ed1-d3b8c9b9e13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:40.635013 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decf209-de04-48fa-9ed1-d3b8c9b9e13e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:40.635325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decf209-de04-48fa-9ed1-d3b8c9b9e13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:40.635597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decf209-de04-48fa-9ed1-d3b8c9b9e13e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:40.636248 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8decf209-de04-48fa-9ed1-d3b8c9b9e13e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:40.637924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8decf209-de04-48fa-9ed1-d3b8c9b9e13e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:40.649960 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1291/2569] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:54:40.654280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d07efe-09c8-470d-acca-879831596e38 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:40.665307 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d07efe-09c8-470d-acca-879831596e38 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:40.670056 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a148d-67bc-4575-915c-7127f0613a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:40.670851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a148d-67bc-4575-915c-7127f0613a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:40.671123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a148d-67bc-4575-915c-7127f0613a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:40.671391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a148d-67bc-4575-915c-7127f0613a48 admin admin] PATH_INFO: `/v3/projects/24846a06a4e1463fa51e8a90f2052225/users/77cd725d0d844bc695244eaa84ec52a8/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:40.686576 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a148d-67bc-4575-915c-7127f0613a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24846a06a4e1463fa51e8a90f2052225, user_id=77cd725d0d844bc695244eaa84ec52a8, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:40.689323 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a148d-67bc-4575-915c-7127f0613a48 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:40.773945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1279/2570] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:54:40 2022] PUT /identity/v3/projects/24846a06a4e1463fa51e8a90f2052225/users/77cd725d0d844bc695244eaa84ec52a8/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:40.785067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f73df7-bdd6-471d-a114-0874c3f267f6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:40.785380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f73df7-bdd6-471d-a114-0874c3f267f6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:40.785653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f73df7-bdd6-471d-a114-0874c3f267f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:40.813538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e2f73df7-bdd6-471d-a114-0874c3f267f6 None None] MFA Rules not processed for user `77cd725d0d844bc695244eaa84ec52a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:40.814796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e2f73df7-bdd6-471d-a114-0874c3f267f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:40.824657 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:40.891792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1292/2571] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:40.896312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a148d-67bc-4575-915c-7127f0613a48 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:40.939144 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:40.958326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a148d-67bc-4575-915c-7127f0613a48 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:40.962983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759a648-f746-442a-85f8-a6932f8a7dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:40.963578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759a648-f746-442a-85f8-a6932f8a7dfc admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:40.963853 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759a648-f746-442a-85f8-a6932f8a7dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:40.964117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759a648-f746-442a-85f8-a6932f8a7dfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:40.964736 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759a648-f746-442a-85f8-a6932f8a7dfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:40.966677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759a648-f746-442a-85f8-a6932f8a7dfc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:41.059127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1280/2572] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:54:40 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:54:41.060598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f73df7-bdd6-471d-a114-0874c3f267f6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:41.072541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f73df7-bdd6-471d-a114-0874c3f267f6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:41.131414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20889db2-adb4-4404-8949-c975bceafa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:41.131935 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20889db2-adb4-4404-8949-c975bceafa00 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:41.132212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20889db2-adb4-4404-8949-c975bceafa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:41.132477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20889db2-adb4-4404-8949-c975bceafa00 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:41.133146 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20889db2-adb4-4404-8949-c975bceafa00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:41.135112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20889db2-adb4-4404-8949-c975bceafa00 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:41.272090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1293/2573] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:54:41 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:41.276285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759a648-f746-442a-85f8-a6932f8a7dfc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:41.287137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759a648-f746-442a-85f8-a6932f8a7dfc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:41.291636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdf667-7d10-40a4-94ee-b70a52439e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:41.292250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdf667-7d10-40a4-94ee-b70a52439e60 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:41.292512 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdf667-7d10-40a4-94ee-b70a52439e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:41.292779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdf667-7d10-40a4-94ee-b70a52439e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:41.293479 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdf667-7d10-40a4-94ee-b70a52439e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:41.295110 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdf667-7d10-40a4-94ee-b70a52439e60 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:41.305309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1281/2574] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:54:41.309746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20889db2-adb4-4404-8949-c975bceafa00 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:41.320433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20889db2-adb4-4404-8949-c975bceafa00 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:41.325097 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e9ed7-1170-4706-bb4f-22560933402f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:41.325751 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e9ed7-1170-4706-bb4f-22560933402f admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:41.326022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e9ed7-1170-4706-bb4f-22560933402f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:41.326322 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e9ed7-1170-4706-bb4f-22560933402f admin admin] PATH_INFO: `/v3/projects/c015c71d61104bca85423161550deade/users/1759c15ec99643b4b1b06234db28678d/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:41.341899 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e9ed7-1170-4706-bb4f-22560933402f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c015c71d61104bca85423161550deade, user_id=1759c15ec99643b4b1b06234db28678d, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:41.344596 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e9ed7-1170-4706-bb4f-22560933402f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:41.458598 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1294/2575] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:54:41 2022] PUT /identity/v3/projects/c015c71d61104bca85423161550deade/users/1759c15ec99643b4b1b06234db28678d/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:41.462766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdf667-7d10-40a4-94ee-b70a52439e60 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:41.472853 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdf667-7d10-40a4-94ee-b70a52439e60 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:41.534418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997c214-82de-4bfb-8f83-3dfcee7c017a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:41.535012 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997c214-82de-4bfb-8f83-3dfcee7c017a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:41.535292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997c214-82de-4bfb-8f83-3dfcee7c017a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:41.535579 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997c214-82de-4bfb-8f83-3dfcee7c017a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:41.536305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997c214-82de-4bfb-8f83-3dfcee7c017a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:41.537952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997c214-82de-4bfb-8f83-3dfcee7c017a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:41.549763 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1282/2576] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:54:41.554289 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e9ed7-1170-4706-bb4f-22560933402f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:41.566573 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e9ed7-1170-4706-bb4f-22560933402f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:41.571608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c58f3-cc2a-4e46-adea-3cd82c34995c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:41.572461 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c58f3-cc2a-4e46-adea-3cd82c34995c admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:41.572741 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c58f3-cc2a-4e46-adea-3cd82c34995c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:41.573016 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c58f3-cc2a-4e46-adea-3cd82c34995c admin admin] PATH_INFO: `/v3/projects/c015c71d61104bca85423161550deade/users/1759c15ec99643b4b1b06234db28678d/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:41.575068 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c58f3-cc2a-4e46-adea-3cd82c34995c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c015c71d61104bca85423161550deade, user_id=1759c15ec99643b4b1b06234db28678d, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:41.577866 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c58f3-cc2a-4e46-adea-3cd82c34995c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:41.589537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1295/2577] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:54:41 2022] PUT /identity/v3/projects/c015c71d61104bca85423161550deade/users/1759c15ec99643b4b1b06234db28678d/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:41.600549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca2a90-75db-4209-bff8-b7ab54e9e402 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:41.601203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca2a90-75db-4209-bff8-b7ab54e9e402 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:41.601475 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca2a90-75db-4209-bff8-b7ab54e9e402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:41.636110 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-50ca2a90-75db-4209-bff8-b7ab54e9e402 None None] MFA Rules not processed for user `1759c15ec99643b4b1b06234db28678d`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:41.640745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-50ca2a90-75db-4209-bff8-b7ab54e9e402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:41.754012 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1283/2578] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:41.767488 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c58f3-cc2a-4e46-adea-3cd82c34995c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:41.769250 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e73c58f3-cc2a-4e46-adea-3cd82c34995c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:41.840588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c58f3-cc2a-4e46-adea-3cd82c34995c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:41.844817 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f93e8d-98ee-450b-a9ea-e52901a7bcd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:41.845414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f93e8d-98ee-450b-a9ea-e52901a7bcd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:41.845685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f93e8d-98ee-450b-a9ea-e52901a7bcd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:41.845950 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f93e8d-98ee-450b-a9ea-e52901a7bcd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:41.855698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f93e8d-98ee-450b-a9ea-e52901a7bcd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:41.857145 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f93e8d-98ee-450b-a9ea-e52901a7bcd2 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:41.862509 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:41.888296 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1296/2579] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:54:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:54:43.275535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca2a90-75db-4209-bff8-b7ab54e9e402 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:43.317193 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca2a90-75db-4209-bff8-b7ab54e9e402 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:43.322303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893ab96-7e30-44c6-9828-301ba34cea0f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1759c15ec99643b4b1b06234db28678d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c015c71d61104bca85423161550deade', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:43.323019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893ab96-7e30-44c6-9828-301ba34cea0f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:43.323305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893ab96-7e30-44c6-9828-301ba34cea0f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:43.323612 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893ab96-7e30-44c6-9828-301ba34cea0f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:43.324265 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893ab96-7e30-44c6-9828-301ba34cea0f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:54:43.324657 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893ab96-7e30-44c6-9828-301ba34cea0f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:43.326375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893ab96-7e30-44c6-9828-301ba34cea0f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:43.335405 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:43.340694 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1284/2580] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:54:43.345618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f93e8d-98ee-450b-a9ea-e52901a7bcd2 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:43.367588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f93e8d-98ee-450b-a9ea-e52901a7bcd2 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:43.372716 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76685116-18a3-4766-b6f5-804bff3d29e9 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1759c15ec99643b4b1b06234db28678d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c015c71d61104bca85423161550deade', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:43.373382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76685116-18a3-4766-b6f5-804bff3d29e9 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:43.373694 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76685116-18a3-4766-b6f5-804bff3d29e9 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:43.373963 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76685116-18a3-4766-b6f5-804bff3d29e9 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:43.374642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76685116-18a3-4766-b6f5-804bff3d29e9 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:43.376580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76685116-18a3-4766-b6f5-804bff3d29e9 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:43.553332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1297/2581] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:54:43 2022] POST /identity/v3/projects => generated 355 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:43.557560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893ab96-7e30-44c6-9828-301ba34cea0f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:43.568265 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893ab96-7e30-44c6-9828-301ba34cea0f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:43.639963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a580101-6761-4416-82f7-f6237509439f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1759c15ec99643b4b1b06234db28678d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c015c71d61104bca85423161550deade', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:43.640956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a580101-6761-4416-82f7-f6237509439f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:43.641236 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a580101-6761-4416-82f7-f6237509439f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:43.641513 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a580101-6761-4416-82f7-f6237509439f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:43.642298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a580101-6761-4416-82f7-f6237509439f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:54:43.642509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a580101-6761-4416-82f7-f6237509439f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:43.644101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a580101-6761-4416-82f7-f6237509439f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:43.660244 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1285/2582] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:54:43.694941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76685116-18a3-4766-b6f5-804bff3d29e9 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:43.706458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76685116-18a3-4766-b6f5-804bff3d29e9 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:43.715671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9742478-a5ef-4462-a8a8-1f3071737172 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1759c15ec99643b4b1b06234db28678d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c015c71d61104bca85423161550deade', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:43.716344 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9742478-a5ef-4462-a8a8-1f3071737172 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:43.716621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9742478-a5ef-4462-a8a8-1f3071737172 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:43.716886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9742478-a5ef-4462-a8a8-1f3071737172 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] PATH_INFO: `/v3/projects/a780eaa2a72946cf9d857d75e018cb18` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:43.718662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9742478-a5ef-4462-a8a8-1f3071737172 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a780eaa2a72946cf9d857d75e018cb18)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:43.720573 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9742478-a5ef-4462-a8a8-1f3071737172 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:43.888373 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9742478-a5ef-4462-a8a8-1f3071737172 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a780eaa2a72946cf9d857d75e018cb18', 'request_id': 'req-c9742478-a5ef-4462-a8a8-1f3071737172'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:44.002418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9742478-a5ef-4462-a8a8-1f3071737172 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a780eaa2a72946cf9d857d75e018cb18', 'request_id': 'req-c9742478-a5ef-4462-a8a8-1f3071737172'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:44.011544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9742478-a5ef-4462-a8a8-1f3071737172 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a780eaa2a72946cf9d857d75e018cb18', 'request_id': 'req-c9742478-a5ef-4462-a8a8-1f3071737172'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:44.025424 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9742478-a5ef-4462-a8a8-1f3071737172 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] The token cache is being invalidate because project a780eaa2a72946cf9d857d75e018cb18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:54:44.025777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9742478-a5ef-4462-a8a8-1f3071737172 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:44.026703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9742478-a5ef-4462-a8a8-1f3071737172 tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:44.028466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1298/2583] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:54:43 2022] DELETE /identity/v3/projects/a780eaa2a72946cf9d857d75e018cb18 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:44.794104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a580101-6761-4416-82f7-f6237509439f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:44.796311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1a580101-6761-4416-82f7-f6237509439f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:44.911311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a580101-6761-4416-82f7-f6237509439f tempest-VolumeQuotaClassesTest-1401945505 tempest-VolumeQuotaClassesTest-1401945505-project-admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:44.973653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:44.974218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:44.974498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:44.974767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] PATH_INFO: `/v3/users/77cd725d0d844bc695244eaa84ec52a8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:44.975953 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77cd725d0d844bc695244eaa84ec52a8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:44.977923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:44.992537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:45.194680 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77cd725d0d844bc695244eaa84ec52a8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:45.207316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77cd725d0d844bc695244eaa84ec52a8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:45.276624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77cd725d0d844bc695244eaa84ec52a8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:45.277255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77cd725d0d844bc695244eaa84ec52a8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:45.285889 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77cd725d0d844bc695244eaa84ec52a8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:45.292652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1286/2584] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:54:44 2022] DELETE /identity/v3/users/77cd725d0d844bc695244eaa84ec52a8 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:45.297938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51212a9-b925-4218-ae10-4ee399977c38 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:45.298527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51212a9-b925-4218-ae10-4ee399977c38 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:45.298527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51212a9-b925-4218-ae10-4ee399977c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:45.336347 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a51212a9-b925-4218-ae10-4ee399977c38 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:45.339288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a51212a9-b925-4218-ae10-4ee399977c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:45.409506 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1299/2585] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:45.419192 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:45.420768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:45.501610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9686846-3999-4d0e-aa7c-a1168b896729 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:45.508106 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4f07d-4274-4e63-91ba-0a8154d1a84a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:45.508790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4f07d-4274-4e63-91ba-0a8154d1a84a service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:45.509010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4f07d-4274-4e63-91ba-0a8154d1a84a service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:45.509269 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4f07d-4274-4e63-91ba-0a8154d1a84a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:45.520347 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4f07d-4274-4e63-91ba-0a8154d1a84a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:45.521790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4f07d-4274-4e63-91ba-0a8154d1a84a service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:45.532736 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1287/2586] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:54:46.211820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51212a9-b925-4218-ae10-4ee399977c38 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:46.213941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a51212a9-b925-4218-ae10-4ee399977c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:46.227489 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51212a9-b925-4218-ae10-4ee399977c38 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:46.232430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:46.233757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:46.233902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:46.234315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] PATH_INFO: `/v3/projects/24846a06a4e1463fa51e8a90f2052225` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:46.235597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=24846a06a4e1463fa51e8a90f2052225)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:46.238434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:46.269023 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:46.359456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24846a06a4e1463fa51e8a90f2052225', 'request_id': 'req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:46.488026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24846a06a4e1463fa51e8a90f2052225', 'request_id': 'req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:46.494759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24846a06a4e1463fa51e8a90f2052225', 'request_id': 'req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:46.508033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] The token cache is being invalidate because project 24846a06a4e1463fa51e8a90f2052225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:54:46.508588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:46.513272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:46.513272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1300/2587] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:54:46 2022] DELETE /identity/v3/projects/24846a06a4e1463fa51e8a90f2052225 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:46.515168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4f07d-4274-4e63-91ba-0a8154d1a84a service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:46.532413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4f07d-4274-4e63-91ba-0a8154d1a84a service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:46.603271 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3daba-ff13-43c5-8ed7-68095d8cb1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:46.607195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3daba-ff13-43c5-8ed7-68095d8cb1e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:46.607195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3daba-ff13-43c5-8ed7-68095d8cb1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:46.607195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3daba-ff13-43c5-8ed7-68095d8cb1e7 admin admin] PATH_INFO: `/v3/users/1759c15ec99643b4b1b06234db28678d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:46.607195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3daba-ff13-43c5-8ed7-68095d8cb1e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1759c15ec99643b4b1b06234db28678d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:46.608725 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3daba-ff13-43c5-8ed7-68095d8cb1e7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:46.638530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:46.989464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cda3daba-ff13-43c5-8ed7-68095d8cb1e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1759c15ec99643b4b1b06234db28678d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:47.003535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cda3daba-ff13-43c5-8ed7-68095d8cb1e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1759c15ec99643b4b1b06234db28678d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:47.105417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cda3daba-ff13-43c5-8ed7-68095d8cb1e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1759c15ec99643b4b1b06234db28678d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:47.106652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cda3daba-ff13-43c5-8ed7-68095d8cb1e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1759c15ec99643b4b1b06234db28678d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:47.114957 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cda3daba-ff13-43c5-8ed7-68095d8cb1e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1759c15ec99643b4b1b06234db28678d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:47.121138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1288/2588] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:54:46 2022] DELETE /identity/v3/users/1759c15ec99643b4b1b06234db28678d => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:47.682616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:47.684223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:47.698954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b902a60-7bd3-4ab9-a3d8-0234e7433c0b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:47.764850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:47.765480 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:47.765752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:47.766020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d admin admin] PATH_INFO: `/v3/projects/c015c71d61104bca85423161550deade` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:47.767175 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c015c71d61104bca85423161550deade)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:47.771064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:47.894634 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c015c71d61104bca85423161550deade', 'request_id': 'req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:47.991004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c015c71d61104bca85423161550deade', 'request_id': 'req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:47.997501 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c015c71d61104bca85423161550deade', 'request_id': 'req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:48.010762 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d admin admin] The token cache is being invalidate because project c015c71d61104bca85423161550deade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:54:48.011060 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:48.011885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dc9f070d-a8fd-42b7-86fa-221c1da87f1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:48.013685 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1301/2589] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:54:47 2022] DELETE /identity/v3/projects/c015c71d61104bca85423161550deade => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:48.019424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84860127-3ec4-404a-959e-3dc20d469c8c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:48.020003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84860127-3ec4-404a-959e-3dc20d469c8c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:48.020003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84860127-3ec4-404a-959e-3dc20d469c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:48.081377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-84860127-3ec4-404a-959e-3dc20d469c8c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:48.082640 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-84860127-3ec4-404a-959e-3dc20d469c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:48.146624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1289/2590] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:48.156204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21081cf9-f916-4c6f-844e-438963393063 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:48.156451 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21081cf9-f916-4c6f-844e-438963393063 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:48.156693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21081cf9-f916-4c6f-844e-438963393063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:48.177235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-21081cf9-f916-4c6f-844e-438963393063 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:48.178563 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-21081cf9-f916-4c6f-844e-438963393063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:48.186780 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1302/2591] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:54:48.192095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84860127-3ec4-404a-959e-3dc20d469c8c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:48.204695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84860127-3ec4-404a-959e-3dc20d469c8c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:48.209675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2fbb3-3b03-4d2a-9bb3-3f1d9a3a14ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:48.210332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2fbb3-3b03-4d2a-9bb3-3f1d9a3a14ab admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:48.210610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2fbb3-3b03-4d2a-9bb3-3f1d9a3a14ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:48.210878 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2fbb3-3b03-4d2a-9bb3-3f1d9a3a14ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:48.211567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2fbb3-3b03-4d2a-9bb3-3f1d9a3a14ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:54:48.211895 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2fbb3-3b03-4d2a-9bb3-3f1d9a3a14ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:48.213478 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2fbb3-3b03-4d2a-9bb3-3f1d9a3a14ab admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:48.228345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1290/2592] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:54:48.233121 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21081cf9-f916-4c6f-844e-438963393063 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:48.243747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21081cf9-f916-4c6f-844e-438963393063 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:48.248641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1e6de-df47-4799-800f-1809c4d6a8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:48.249250 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1e6de-df47-4799-800f-1809c4d6a8ac admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:48.249526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1e6de-df47-4799-800f-1809c4d6a8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:48.249791 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1e6de-df47-4799-800f-1809c4d6a8ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:48.250519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1e6de-df47-4799-800f-1809c4d6a8ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:48.252453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1e6de-df47-4799-800f-1809c4d6a8ac admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:48.358847 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1303/2593] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:54:48 2022] POST /identity/v3/projects => generated 393 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:54:48.363738 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2fbb3-3b03-4d2a-9bb3-3f1d9a3a14ab admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:48.374857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2fbb3-3b03-4d2a-9bb3-3f1d9a3a14ab admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:48.438282 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd4667-19e2-4485-9cfb-272b8c85df51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:48.438282 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd4667-19e2-4485-9cfb-272b8c85df51 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:48.438282 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd4667-19e2-4485-9cfb-272b8c85df51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:48.438282 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd4667-19e2-4485-9cfb-272b8c85df51 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:48.438655 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd4667-19e2-4485-9cfb-272b8c85df51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:48.439502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd4667-19e2-4485-9cfb-272b8c85df51 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:48.447767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:48.539076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1291/2594] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:54:48 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:48.544274 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1e6de-df47-4799-800f-1809c4d6a8ac admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:48.562969 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1e6de-df47-4799-800f-1809c4d6a8ac admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:48.569338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b60447-bd5f-4d48-961c-7dcdb484657b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:48.569961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b60447-bd5f-4d48-961c-7dcdb484657b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:48.570278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b60447-bd5f-4d48-961c-7dcdb484657b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:48.570553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b60447-bd5f-4d48-961c-7dcdb484657b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:48.571212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b60447-bd5f-4d48-961c-7dcdb484657b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:48.572842 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b60447-bd5f-4d48-961c-7dcdb484657b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:48.584119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1304/2595] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:54:48.593400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd4667-19e2-4485-9cfb-272b8c85df51 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:48.609114 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd4667-19e2-4485-9cfb-272b8c85df51 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:48.618355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fd655-5d64-43bb-ae3c-67e6ae7a2867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:48.618736 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fd655-5d64-43bb-ae3c-67e6ae7a2867 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:48.619019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fd655-5d64-43bb-ae3c-67e6ae7a2867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:48.619286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fd655-5d64-43bb-ae3c-67e6ae7a2867 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:48.619946 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fd655-5d64-43bb-ae3c-67e6ae7a2867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:48.621722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fd655-5d64-43bb-ae3c-67e6ae7a2867 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:48.635146 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1292/2596] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:54:48.639796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b60447-bd5f-4d48-961c-7dcdb484657b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:48.646643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:48.652543 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b60447-bd5f-4d48-961c-7dcdb484657b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:48.657990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131ced0-7c33-496a-977a-27f1f0da7f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:48.658787 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131ced0-7c33-496a-977a-27f1f0da7f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:48.659120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131ced0-7c33-496a-977a-27f1f0da7f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:48.659482 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131ced0-7c33-496a-977a-27f1f0da7f02 admin admin] PATH_INFO: `/v3/projects/0994a1d40f9b4de2b4911c8175e29ed1/users/c106eec6abaa4bfaa90345ee70624b4e/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:48.684699 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131ced0-7c33-496a-977a-27f1f0da7f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0994a1d40f9b4de2b4911c8175e29ed1, user_id=c106eec6abaa4bfaa90345ee70624b4e, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:48.688774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131ced0-7c33-496a-977a-27f1f0da7f02 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:48.784515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1305/2597] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:54:48 2022] PUT /identity/v3/projects/0994a1d40f9b4de2b4911c8175e29ed1/users/c106eec6abaa4bfaa90345ee70624b4e/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:48.795038 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4169f0a-d891-4206-9cb4-5478e92cfaa7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:48.795319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4169f0a-d891-4206-9cb4-5478e92cfaa7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:48.795591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4169f0a-d891-4206-9cb4-5478e92cfaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:48.950481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a4169f0a-d891-4206-9cb4-5478e92cfaa7 None None] MFA Rules not processed for user `c106eec6abaa4bfaa90345ee70624b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:48.951942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a4169f0a-d891-4206-9cb4-5478e92cfaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:49.028390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1293/2598] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:49.033103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131ced0-7c33-496a-977a-27f1f0da7f02 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:49.100544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131ced0-7c33-496a-977a-27f1f0da7f02 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:49.107002 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164c2d9-de5d-484e-a3a7-c6fc0b7b49be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:49.108853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164c2d9-de5d-484e-a3a7-c6fc0b7b49be admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:49.109132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164c2d9-de5d-484e-a3a7-c6fc0b7b49be admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:49.109726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164c2d9-de5d-484e-a3a7-c6fc0b7b49be admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:49.110708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164c2d9-de5d-484e-a3a7-c6fc0b7b49be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:49.112658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164c2d9-de5d-484e-a3a7-c6fc0b7b49be admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:49.261112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1306/2599] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:54:49 2022] POST /identity/v3/projects => generated 393 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:54:49.265324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4169f0a-d891-4206-9cb4-5478e92cfaa7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:49.276407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4169f0a-d891-4206-9cb4-5478e92cfaa7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:49.337921 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba84f2-743d-43f7-9ef1-1ab2f5c78796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:49.338601 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba84f2-743d-43f7-9ef1-1ab2f5c78796 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:49.338950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba84f2-743d-43f7-9ef1-1ab2f5c78796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:49.339234 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba84f2-743d-43f7-9ef1-1ab2f5c78796 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:49.339987 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba84f2-743d-43f7-9ef1-1ab2f5c78796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:49.342814 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba84f2-743d-43f7-9ef1-1ab2f5c78796 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:49.437101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1294/2600] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:54:49 2022] POST /identity/v3/users => generated 303 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:49.441647 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164c2d9-de5d-484e-a3a7-c6fc0b7b49be admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:49.452778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164c2d9-de5d-484e-a3a7-c6fc0b7b49be admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:49.457449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ac667-255d-4062-9851-6efd1a9385cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:49.458057 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ac667-255d-4062-9851-6efd1a9385cb admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:49.458389 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ac667-255d-4062-9851-6efd1a9385cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:49.458679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ac667-255d-4062-9851-6efd1a9385cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:49.459316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ac667-255d-4062-9851-6efd1a9385cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:49.460922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ac667-255d-4062-9851-6efd1a9385cb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:49.474062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1307/2601] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:54:49.478932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba84f2-743d-43f7-9ef1-1ab2f5c78796 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:49.490398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba84f2-743d-43f7-9ef1-1ab2f5c78796 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:49.495034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc6e2f-d889-40c7-9a88-29aadcf220b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:49.495685 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc6e2f-d889-40c7-9a88-29aadcf220b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:49.495955 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc6e2f-d889-40c7-9a88-29aadcf220b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:49.496218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc6e2f-d889-40c7-9a88-29aadcf220b2 admin admin] PATH_INFO: `/v3/projects/284cb21f00f24fc0a834f39e551bbb6a/users/22e0f52f72774b71b5394d85efcdafa0/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:49.512141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc6e2f-d889-40c7-9a88-29aadcf220b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=284cb21f00f24fc0a834f39e551bbb6a, user_id=22e0f52f72774b71b5394d85efcdafa0, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:49.514706 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc6e2f-d889-40c7-9a88-29aadcf220b2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:49.622057 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1295/2602] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:54:49 2022] PUT /identity/v3/projects/284cb21f00f24fc0a834f39e551bbb6a/users/22e0f52f72774b71b5394d85efcdafa0/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:49.626679 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ac667-255d-4062-9851-6efd1a9385cb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:49.637204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ac667-255d-4062-9851-6efd1a9385cb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:49.703105 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a1527-cb26-4eb0-8e0f-548ee9340a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:49.703720 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a1527-cb26-4eb0-8e0f-548ee9340a25 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:49.703992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a1527-cb26-4eb0-8e0f-548ee9340a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:49.704259 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a1527-cb26-4eb0-8e0f-548ee9340a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:49.704903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a1527-cb26-4eb0-8e0f-548ee9340a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:49.706534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a1527-cb26-4eb0-8e0f-548ee9340a25 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:49.717228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1308/2603] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:54:49.721486 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc6e2f-d889-40c7-9a88-29aadcf220b2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:49.731966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc6e2f-d889-40c7-9a88-29aadcf220b2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:49.737812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29149ff-be4c-497a-a886-ec716a871e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:49.738489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29149ff-be4c-497a-a886-ec716a871e2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:49.738758 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29149ff-be4c-497a-a886-ec716a871e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:49.739023 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29149ff-be4c-497a-a886-ec716a871e2c admin admin] PATH_INFO: `/v3/projects/284cb21f00f24fc0a834f39e551bbb6a/users/22e0f52f72774b71b5394d85efcdafa0/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:49.740914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29149ff-be4c-497a-a886-ec716a871e2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=284cb21f00f24fc0a834f39e551bbb6a, user_id=22e0f52f72774b71b5394d85efcdafa0, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:49.743542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29149ff-be4c-497a-a886-ec716a871e2c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:49.757913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1296/2604] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:54:49 2022] PUT /identity/v3/projects/284cb21f00f24fc0a834f39e551bbb6a/users/22e0f52f72774b71b5394d85efcdafa0/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:54:49.770445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf1e1a-e925-4bd6-beaf-e7ebb5008bd1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:49.770723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf1e1a-e925-4bd6-beaf-e7ebb5008bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:49.771456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf1e1a-e925-4bd6-beaf-e7ebb5008bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:49.815341 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9baf1e1a-e925-4bd6-beaf-e7ebb5008bd1 None None] MFA Rules not processed for user `22e0f52f72774b71b5394d85efcdafa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:49.817555 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9baf1e1a-e925-4bd6-beaf-e7ebb5008bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:49.875459 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:49.923564 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1309/2605] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:49 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:49.934709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29149ff-be4c-497a-a886-ec716a871e2c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:49.937009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a29149ff-be4c-497a-a886-ec716a871e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:49.963618 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:50.008997 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29149ff-be4c-497a-a886-ec716a871e2c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:50.012751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07e062-cbdc-4196-9ffb-1bf475ba3e8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:50.013357 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07e062-cbdc-4196-9ffb-1bf475ba3e8e service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:50.013637 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07e062-cbdc-4196-9ffb-1bf475ba3e8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:50.013902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07e062-cbdc-4196-9ffb-1bf475ba3e8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:50.023712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07e062-cbdc-4196-9ffb-1bf475ba3e8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:50.025274 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07e062-cbdc-4196-9ffb-1bf475ba3e8e service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:50.036007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1297/2606] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:54:49 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:54:50.083954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf1e1a-e925-4bd6-beaf-e7ebb5008bd1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:50.085411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9baf1e1a-e925-4bd6-beaf-e7ebb5008bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:50.098317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf1e1a-e925-4bd6-beaf-e7ebb5008bd1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:50.101900 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541b33e-f654-4191-8d16-8c853ca3453d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:50.102527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541b33e-f654-4191-8d16-8c853ca3453d service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:50.102801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541b33e-f654-4191-8d16-8c853ca3453d service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:50.103062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541b33e-f654-4191-8d16-8c853ca3453d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:50.160678 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541b33e-f654-4191-8d16-8c853ca3453d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:50.162044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541b33e-f654-4191-8d16-8c853ca3453d service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:50.172228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1310/2607] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:54:50 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:54:53.633323 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07e062-cbdc-4196-9ffb-1bf475ba3e8e service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:53.634883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c07e062-cbdc-4196-9ffb-1bf475ba3e8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:53.749680 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07e062-cbdc-4196-9ffb-1bf475ba3e8e service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:53.771821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79730bc0-1e88-4bbd-8400-7235da310450 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d03db170fb6460794e32669456bcdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:53.772987 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79730bc0-1e88-4bbd-8400-7235da310450 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:53.773464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79730bc0-1e88-4bbd-8400-7235da310450 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:53.773943 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79730bc0-1e88-4bbd-8400-7235da310450 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:53.777669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-79730bc0-1e88-4bbd-8400-7235da310450 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:53.802724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79730bc0-1e88-4bbd-8400-7235da310450 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:53.802724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79730bc0-1e88-4bbd-8400-7235da310450 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:53.812782 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1298/2608] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:54:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:54:55.590390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541b33e-f654-4191-8d16-8c853ca3453d service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:55.706517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541b33e-f654-4191-8d16-8c853ca3453d service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:55.714968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:55.714968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:55.714968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:55.714968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] PATH_INFO: `/v3/users/c106eec6abaa4bfaa90345ee70624b4e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:55.718704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c106eec6abaa4bfaa90345ee70624b4e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:55.721237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:55.937193 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c106eec6abaa4bfaa90345ee70624b4e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:55.938032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c106eec6abaa4bfaa90345ee70624b4e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:56.047395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c106eec6abaa4bfaa90345ee70624b4e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:56.063307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c106eec6abaa4bfaa90345ee70624b4e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:56.080926 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:56.089889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c106eec6abaa4bfaa90345ee70624b4e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:56.099378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1311/2609] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:54:55 2022] DELETE /identity/v3/users/c106eec6abaa4bfaa90345ee70624b4e => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:56.104816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f184d-82d2-4ab1-b875-bb16eb80c02a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:56.105459 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f184d-82d2-4ab1-b875-bb16eb80c02a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:56.105622 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f184d-82d2-4ab1-b875-bb16eb80c02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:56.135112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2b2f184d-82d2-4ab1-b875-bb16eb80c02a None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:56.136630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2b2f184d-82d2-4ab1-b875-bb16eb80c02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:56.201075 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1299/2610] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:54:56.211593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:56.213191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:56.292449 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8194eb4-7088-4e71-a296-8f619426dc0b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:56.297019 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffa5d0-a12a-4ea3-af7e-54e43f8ca1a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:56.297639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffa5d0-a12a-4ea3-af7e-54e43f8ca1a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:56.297905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffa5d0-a12a-4ea3-af7e-54e43f8ca1a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:56.298195 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffa5d0-a12a-4ea3-af7e-54e43f8ca1a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:56.311616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffa5d0-a12a-4ea3-af7e-54e43f8ca1a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:56.313196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffa5d0-a12a-4ea3-af7e-54e43f8ca1a9 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:56.327228 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1312/2611] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:54:56.849824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f184d-82d2-4ab1-b875-bb16eb80c02a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:56.851380 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2b2f184d-82d2-4ab1-b875-bb16eb80c02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:56.864471 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f184d-82d2-4ab1-b875-bb16eb80c02a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:56.870077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:56.870766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:56.871045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:56.871315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] PATH_INFO: `/v3/projects/0994a1d40f9b4de2b4911c8175e29ed1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:56.872494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] RBAC: Authorizing `identity:delete_project(project_id=0994a1d40f9b4de2b4911c8175e29ed1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:56.874979 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:56.997387 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:57.019919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0994a1d40f9b4de2b4911c8175e29ed1', 'request_id': 'req-91ff0b90-f085-462d-a93b-c9951a8deded'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:57.117510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0994a1d40f9b4de2b4911c8175e29ed1', 'request_id': 'req-91ff0b90-f085-462d-a93b-c9951a8deded'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:57.124102 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0994a1d40f9b4de2b4911c8175e29ed1', 'request_id': 'req-91ff0b90-f085-462d-a93b-c9951a8deded'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:57.135781 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] The token cache is being invalidate because project 0994a1d40f9b4de2b4911c8175e29ed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:54:57.136071 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:57.136718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:57.138272 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1300/2612] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:54:56 2022] DELETE /identity/v3/projects/0994a1d40f9b4de2b4911c8175e29ed1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:57.142443 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffa5d0-a12a-4ea3-af7e-54e43f8ca1a9 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:57.152161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffa5d0-a12a-4ea3-af7e-54e43f8ca1a9 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:57.211796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ec587-4688-46db-9052-5db3e73a057e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:57.214463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ec587-4688-46db-9052-5db3e73a057e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:57.214739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ec587-4688-46db-9052-5db3e73a057e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:57.215011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ec587-4688-46db-9052-5db3e73a057e admin admin] PATH_INFO: `/v3/users/22e0f52f72774b71b5394d85efcdafa0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:57.216286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ec587-4688-46db-9052-5db3e73a057e admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e0f52f72774b71b5394d85efcdafa0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:57.218408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ec587-4688-46db-9052-5db3e73a057e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:57.475947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-772ec587-4688-46db-9052-5db3e73a057e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e0f52f72774b71b5394d85efcdafa0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:57.476640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-772ec587-4688-46db-9052-5db3e73a057e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e0f52f72774b71b5394d85efcdafa0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:57.590655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-772ec587-4688-46db-9052-5db3e73a057e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e0f52f72774b71b5394d85efcdafa0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:57.599435 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-772ec587-4688-46db-9052-5db3e73a057e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e0f52f72774b71b5394d85efcdafa0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:57.613037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-772ec587-4688-46db-9052-5db3e73a057e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e0f52f72774b71b5394d85efcdafa0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:57.620084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1313/2613] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:54:57 2022] DELETE /identity/v3/users/22e0f52f72774b71b5394d85efcdafa0 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:58.102324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:58.104220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:58.120180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff0b90-f085-462d-a93b-c9951a8deded admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:58.178130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b6909-5732-43f0-9377-8b8cd22cbf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:58.182501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b6909-5732-43f0-9377-8b8cd22cbf4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:58.182501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b6909-5732-43f0-9377-8b8cd22cbf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:58.182501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b6909-5732-43f0-9377-8b8cd22cbf4f admin admin] PATH_INFO: `/v3/projects/284cb21f00f24fc0a834f39e551bbb6a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:58.182501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b6909-5732-43f0-9377-8b8cd22cbf4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=284cb21f00f24fc0a834f39e551bbb6a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:58.182501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b6909-5732-43f0-9377-8b8cd22cbf4f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:58.302780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c60b6909-5732-43f0-9377-8b8cd22cbf4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '284cb21f00f24fc0a834f39e551bbb6a', 'request_id': 'req-c60b6909-5732-43f0-9377-8b8cd22cbf4f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:58.417650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c60b6909-5732-43f0-9377-8b8cd22cbf4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '284cb21f00f24fc0a834f39e551bbb6a', 'request_id': 'req-c60b6909-5732-43f0-9377-8b8cd22cbf4f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:58.425071 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c60b6909-5732-43f0-9377-8b8cd22cbf4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '284cb21f00f24fc0a834f39e551bbb6a', 'request_id': 'req-c60b6909-5732-43f0-9377-8b8cd22cbf4f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:58.437203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c60b6909-5732-43f0-9377-8b8cd22cbf4f admin admin] The token cache is being invalidate because project 284cb21f00f24fc0a834f39e551bbb6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:54:58.437449 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c60b6909-5732-43f0-9377-8b8cd22cbf4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:58.438169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c60b6909-5732-43f0-9377-8b8cd22cbf4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:54:58.439704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1301/2614] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:54:58 2022] DELETE /identity/v3/projects/284cb21f00f24fc0a834f39e551bbb6a => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:58.444695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd7d77-2f1b-4201-b259-2cdb8a47eb53 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:58.445229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd7d77-2f1b-4201-b259-2cdb8a47eb53 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:58.445229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd7d77-2f1b-4201-b259-2cdb8a47eb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:58.466378 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2bbd7d77-2f1b-4201-b259-2cdb8a47eb53 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:58.467641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2bbd7d77-2f1b-4201-b259-2cdb8a47eb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:58.477818 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:58.534562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1314/2615] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:58.542784 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f1ce-be23-44d9-acbe-db609e232da1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:58.543062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f1ce-be23-44d9-acbe-db609e232da1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:58.543465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f1ce-be23-44d9-acbe-db609e232da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:58.565562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8ad7f1ce-be23-44d9-acbe-db609e232da1 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:58.566890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8ad7f1ce-be23-44d9-acbe-db609e232da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:58.575708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1302/2616] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:58.580297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd7d77-2f1b-4201-b259-2cdb8a47eb53 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:58.590430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd7d77-2f1b-4201-b259-2cdb8a47eb53 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:58.595828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b838be-5511-4244-a7b6-bd0be875caf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:58.596517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b838be-5511-4244-a7b6-bd0be875caf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:58.596856 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b838be-5511-4244-a7b6-bd0be875caf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:58.597179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b838be-5511-4244-a7b6-bd0be875caf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:58.597839 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b838be-5511-4244-a7b6-bd0be875caf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:54:58.598138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b838be-5511-4244-a7b6-bd0be875caf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:58.599834 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b838be-5511-4244-a7b6-bd0be875caf6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:58.614732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1315/2617] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:54:58.619119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7f1ce-be23-44d9-acbe-db609e232da1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:58.629064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7f1ce-be23-44d9-acbe-db609e232da1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:58.634023 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf6ef8-8c9a-49e3-9250-8ac426f11bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:58.634692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf6ef8-8c9a-49e3-9250-8ac426f11bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:58.634985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf6ef8-8c9a-49e3-9250-8ac426f11bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:58.635304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf6ef8-8c9a-49e3-9250-8ac426f11bf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:58.636098 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf6ef8-8c9a-49e3-9250-8ac426f11bf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:58.637997 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abf6ef8-8c9a-49e3-9250-8ac426f11bf5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:58.747383 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1303/2618] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:54:58 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:58.751588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b838be-5511-4244-a7b6-bd0be875caf6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:58.762061 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b838be-5511-4244-a7b6-bd0be875caf6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:58.824466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dd12d-3c08-4dc4-adf1-f07803f14a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:58.825100 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dd12d-3c08-4dc4-adf1-f07803f14a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:58.825371 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dd12d-3c08-4dc4-adf1-f07803f14a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:58.825637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dd12d-3c08-4dc4-adf1-f07803f14a4f admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:58.826307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dd12d-3c08-4dc4-adf1-f07803f14a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:58.828843 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dd12d-3c08-4dc4-adf1-f07803f14a4f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:58.975699 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1316/2619] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:54:58 2022] POST /identity/v3/users => generated 295 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:58.981162 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf6ef8-8c9a-49e3-9250-8ac426f11bf5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:59.000273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf6ef8-8c9a-49e3-9250-8ac426f11bf5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:59.005356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a2138-7360-4739-9d79-47d52ed3a700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:59.006030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a2138-7360-4739-9d79-47d52ed3a700 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:59.006355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a2138-7360-4739-9d79-47d52ed3a700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:59.006610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a2138-7360-4739-9d79-47d52ed3a700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:59.007284 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a2138-7360-4739-9d79-47d52ed3a700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:59.008923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a2138-7360-4739-9d79-47d52ed3a700 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:59.025874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1304/2620] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:54:59.030867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dd12d-3c08-4dc4-adf1-f07803f14a4f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:59.041803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dd12d-3c08-4dc4-adf1-f07803f14a4f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:59.046803 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840d636-0874-4ab9-ad76-842ad5696931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:59.047553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840d636-0874-4ab9-ad76-842ad5696931 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:59.047910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840d636-0874-4ab9-ad76-842ad5696931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:59.048441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840d636-0874-4ab9-ad76-842ad5696931 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:59.049179 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840d636-0874-4ab9-ad76-842ad5696931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:59.051352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840d636-0874-4ab9-ad76-842ad5696931 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:59.187997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1317/2621] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:54:59.194260 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a2138-7360-4739-9d79-47d52ed3a700 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:59.212031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a2138-7360-4739-9d79-47d52ed3a700 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:59.220280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216f80a-03ff-4a56-92c7-9509c6c4c3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:59.221419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216f80a-03ff-4a56-92c7-9509c6c4c3d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:59.221849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216f80a-03ff-4a56-92c7-9509c6c4c3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:59.222173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216f80a-03ff-4a56-92c7-9509c6c4c3d4 admin admin] PATH_INFO: `/v3/projects/8785e375ef1440c39fc9e15154df12f1/users/3ed52e2d368148caaf19c3fee7fef878/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:59.226426 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:59.241836 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216f80a-03ff-4a56-92c7-9509c6c4c3d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8785e375ef1440c39fc9e15154df12f1, user_id=3ed52e2d368148caaf19c3fee7fef878, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:59.246700 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216f80a-03ff-4a56-92c7-9509c6c4c3d4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:59.357787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1305/2622] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:54:59 2022] PUT /identity/v3/projects/8785e375ef1440c39fc9e15154df12f1/users/3ed52e2d368148caaf19c3fee7fef878/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:54:59.369537 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd099e8-fb37-46ab-aa0c-636ad2724e59 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:59.369808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd099e8-fb37-46ab-aa0c-636ad2724e59 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:59.370074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd099e8-fb37-46ab-aa0c-636ad2724e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:59.403364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fcd099e8-fb37-46ab-aa0c-636ad2724e59 None None] MFA Rules not processed for user `3ed52e2d368148caaf19c3fee7fef878`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:54:59.404625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fcd099e8-fb37-46ab-aa0c-636ad2724e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:54:59.480412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1318/2623] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:54:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:54:59.486069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216f80a-03ff-4a56-92c7-9509c6c4c3d4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:59.568162 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216f80a-03ff-4a56-92c7-9509c6c4c3d4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:59.573309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526805a5-7713-42c6-8e58-bd84d319d584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:59.573935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526805a5-7713-42c6-8e58-bd84d319d584 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:59.574242 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526805a5-7713-42c6-8e58-bd84d319d584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:59.574507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526805a5-7713-42c6-8e58-bd84d319d584 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:59.575163 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526805a5-7713-42c6-8e58-bd84d319d584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:59.577290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526805a5-7713-42c6-8e58-bd84d319d584 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:59.661948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1306/2624] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:54:59 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:59.671563 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd099e8-fb37-46ab-aa0c-636ad2724e59 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:59.682812 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd099e8-fb37-46ab-aa0c-636ad2724e59 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:59.687930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:54:59.755286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69797d-f0aa-4a49-878e-5994ff469acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:59.755950 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69797d-f0aa-4a49-878e-5994ff469acf admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:59.756208 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69797d-f0aa-4a49-878e-5994ff469acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:59.756492 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69797d-f0aa-4a49-878e-5994ff469acf admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:59.757136 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69797d-f0aa-4a49-878e-5994ff469acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:59.759163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69797d-f0aa-4a49-878e-5994ff469acf admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:59.849764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1319/2625] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:54:59 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:54:59.854141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526805a5-7713-42c6-8e58-bd84d319d584 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:59.866541 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526805a5-7713-42c6-8e58-bd84d319d584 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:59.871391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8633f2-cb8a-4449-857a-1c7ec41ca17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:59.872028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8633f2-cb8a-4449-857a-1c7ec41ca17e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:59.872413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8633f2-cb8a-4449-857a-1c7ec41ca17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:59.872684 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8633f2-cb8a-4449-857a-1c7ec41ca17e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:59.873356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8633f2-cb8a-4449-857a-1c7ec41ca17e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:59.875661 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8633f2-cb8a-4449-857a-1c7ec41ca17e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:54:59.887342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1307/2626] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:54:59.893791 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69797d-f0aa-4a49-878e-5994ff469acf admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:54:59.906052 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69797d-f0aa-4a49-878e-5994ff469acf admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:54:59.911639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7246e-086d-4787-8497-0a181297cd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:54:59.912375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7246e-086d-4787-8497-0a181297cd44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:54:59.912699 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7246e-086d-4787-8497-0a181297cd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:54:59.912975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7246e-086d-4787-8497-0a181297cd44 admin admin] PATH_INFO: `/v3/projects/6d6cff138b4e42c5822c7fc6abef77aa/users/574a957026a34c619e4a696ceff85c20/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:54:59.930823 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7246e-086d-4787-8497-0a181297cd44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d6cff138b4e42c5822c7fc6abef77aa, user_id=574a957026a34c619e4a696ceff85c20, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:54:59.933541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7246e-086d-4787-8497-0a181297cd44 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:00.048690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1320/2627] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:54:59 2022] PUT /identity/v3/projects/6d6cff138b4e42c5822c7fc6abef77aa/users/574a957026a34c619e4a696ceff85c20/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:00.053160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8633f2-cb8a-4449-857a-1c7ec41ca17e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:00.064901 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8633f2-cb8a-4449-857a-1c7ec41ca17e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:00.123905 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1122d1b-1a3a-45e0-ad48-bac8ace667ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:00.125133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1122d1b-1a3a-45e0-ad48-bac8ace667ac admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:00.125437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1122d1b-1a3a-45e0-ad48-bac8ace667ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:00.125676 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1122d1b-1a3a-45e0-ad48-bac8ace667ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:00.126620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1122d1b-1a3a-45e0-ad48-bac8ace667ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:00.128252 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1122d1b-1a3a-45e0-ad48-bac8ace667ac admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:00.138936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1308/2628] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:00.143382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7246e-086d-4787-8497-0a181297cd44 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:00.154115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7246e-086d-4787-8497-0a181297cd44 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:00.158842 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86010d-c48e-410b-9976-b20504dafb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:00.159494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86010d-c48e-410b-9976-b20504dafb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:00.159764 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86010d-c48e-410b-9976-b20504dafb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:00.160034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86010d-c48e-410b-9976-b20504dafb86 admin admin] PATH_INFO: `/v3/projects/6d6cff138b4e42c5822c7fc6abef77aa/users/574a957026a34c619e4a696ceff85c20/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:00.161927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86010d-c48e-410b-9976-b20504dafb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d6cff138b4e42c5822c7fc6abef77aa, user_id=574a957026a34c619e4a696ceff85c20, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:00.164644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86010d-c48e-410b-9976-b20504dafb86 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:00.176408 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1321/2629] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:00 2022] PUT /identity/v3/projects/6d6cff138b4e42c5822c7fc6abef77aa/users/574a957026a34c619e4a696ceff85c20/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:00.188919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6756f54-5273-4526-be5e-fb8898433b76 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:00.189216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6756f54-5273-4526-be5e-fb8898433b76 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:00.189464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6756f54-5273-4526-be5e-fb8898433b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:00.220278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f6756f54-5273-4526-be5e-fb8898433b76 None None] MFA Rules not processed for user `574a957026a34c619e4a696ceff85c20`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:00.221518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6756f54-5273-4526-be5e-fb8898433b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:00.276956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:00.323752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1309/2630] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:00.334779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86010d-c48e-410b-9976-b20504dafb86 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:00.336559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bc86010d-c48e-410b-9976-b20504dafb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:00.399474 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86010d-c48e-410b-9976-b20504dafb86 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:00.403351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1b89a-bdd2-4aa8-88ab-977e380cb7a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:00.404024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1b89a-bdd2-4aa8-88ab-977e380cb7a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:00.404321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1b89a-bdd2-4aa8-88ab-977e380cb7a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:00.404590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1b89a-bdd2-4aa8-88ab-977e380cb7a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:00.414746 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1b89a-bdd2-4aa8-88ab-977e380cb7a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:00.416129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1b89a-bdd2-4aa8-88ab-977e380cb7a8 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:00.427177 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1322/2631] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:55:00.576318 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6756f54-5273-4526-be5e-fb8898433b76 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:00.646891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6756f54-5273-4526-be5e-fb8898433b76 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:00.651571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:00.652245 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:00.652514 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:00.652800 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] PATH_INFO: `/v3/users/3ed52e2d368148caaf19c3fee7fef878` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:00.653937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ed52e2d368148caaf19c3fee7fef878)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:00.656492 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:00.899838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ed52e2d368148caaf19c3fee7fef878'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:00.916626 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ed52e2d368148caaf19c3fee7fef878'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:01.007880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ed52e2d368148caaf19c3fee7fef878'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:01.008745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ed52e2d368148caaf19c3fee7fef878'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:01.019411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ed52e2d368148caaf19c3fee7fef878'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:01.028042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1310/2632] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:55:00 2022] DELETE /identity/v3/users/3ed52e2d368148caaf19c3fee7fef878 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:01.036089 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d784b-2c7f-4e76-9f6b-c67a1105eb7b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:01.036633 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d784b-2c7f-4e76-9f6b-c67a1105eb7b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:01.036976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d784b-2c7f-4e76-9f6b-c67a1105eb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:01.064243 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-586d784b-2c7f-4e76-9f6b-c67a1105eb7b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:01.065571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-586d784b-2c7f-4e76-9f6b-c67a1105eb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:01.126697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1323/2633] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:01.136692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:01.138111 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:01.205702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:01.226557 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db816a6-b441-47d7-82d1-1ce4a1c5fb14 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:01.230966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d88bd3-fdc7-4085-9bbf-75381d73d8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:01.231705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d88bd3-fdc7-4085-9bbf-75381d73d8af service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:01.231859 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d88bd3-fdc7-4085-9bbf-75381d73d8af service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:01.232931 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d88bd3-fdc7-4085-9bbf-75381d73d8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:01.247170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d88bd3-fdc7-4085-9bbf-75381d73d8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:01.248823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d88bd3-fdc7-4085-9bbf-75381d73d8af service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:01.259211 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1311/2634] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:55:01.758359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d784b-2c7f-4e76-9f6b-c67a1105eb7b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:01.759985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-586d784b-2c7f-4e76-9f6b-c67a1105eb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:01.778017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:01.778017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d784b-2c7f-4e76-9f6b-c67a1105eb7b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:01.778959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:01.779599 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:01.780467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:01.780742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] PATH_INFO: `/v3/projects/8785e375ef1440c39fc9e15154df12f1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:01.782030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=8785e375ef1440c39fc9e15154df12f1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:01.784103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:01.957691 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8785e375ef1440c39fc9e15154df12f1', 'request_id': 'req-3a7ca978-2803-407d-8ef5-74edd41f8fce'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:02.048446 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8785e375ef1440c39fc9e15154df12f1', 'request_id': 'req-3a7ca978-2803-407d-8ef5-74edd41f8fce'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:02.055684 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8785e375ef1440c39fc9e15154df12f1', 'request_id': 'req-3a7ca978-2803-407d-8ef5-74edd41f8fce'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:02.068530 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] The token cache is being invalidate because project 8785e375ef1440c39fc9e15154df12f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:55:02.069009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:02.069688 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:02.071328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1324/2635] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:55:01 2022] DELETE /identity/v3/projects/8785e375ef1440c39fc9e15154df12f1 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:02.075366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d88bd3-fdc7-4085-9bbf-75381d73d8af service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:02.085248 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d88bd3-fdc7-4085-9bbf-75381d73d8af service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:02.168077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf368e5-01c7-49dc-9048-290c03ebc3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:02.168630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf368e5-01c7-49dc-9048-290c03ebc3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:02.168914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf368e5-01c7-49dc-9048-290c03ebc3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:02.169257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf368e5-01c7-49dc-9048-290c03ebc3a6 admin admin] PATH_INFO: `/v3/users/574a957026a34c619e4a696ceff85c20` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:02.170455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf368e5-01c7-49dc-9048-290c03ebc3a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=574a957026a34c619e4a696ceff85c20)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:02.173042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf368e5-01c7-49dc-9048-290c03ebc3a6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:02.399611 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6bf368e5-01c7-49dc-9048-290c03ebc3a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '574a957026a34c619e4a696ceff85c20'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:02.414008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6bf368e5-01c7-49dc-9048-290c03ebc3a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '574a957026a34c619e4a696ceff85c20'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:02.525965 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6bf368e5-01c7-49dc-9048-290c03ebc3a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '574a957026a34c619e4a696ceff85c20'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:02.526716 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6bf368e5-01c7-49dc-9048-290c03ebc3a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '574a957026a34c619e4a696ceff85c20'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:02.535603 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6bf368e5-01c7-49dc-9048-290c03ebc3a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '574a957026a34c619e4a696ceff85c20'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:02.541701 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1312/2636] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:55:02 2022] DELETE /identity/v3/users/574a957026a34c619e4a696ceff85c20 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:03.049511 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:03.051213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:03.065711 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ca978-2803-407d-8ef5-74edd41f8fce admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:03.147608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df77ab9-93bf-404e-b302-f48d98a9c6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:03.148252 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df77ab9-93bf-404e-b302-f48d98a9c6d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:03.148522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df77ab9-93bf-404e-b302-f48d98a9c6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:03.148919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df77ab9-93bf-404e-b302-f48d98a9c6d4 admin admin] PATH_INFO: `/v3/projects/6d6cff138b4e42c5822c7fc6abef77aa` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:03.151407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df77ab9-93bf-404e-b302-f48d98a9c6d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d6cff138b4e42c5822c7fc6abef77aa)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:03.153337 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df77ab9-93bf-404e-b302-f48d98a9c6d4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:03.371955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7df77ab9-93bf-404e-b302-f48d98a9c6d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d6cff138b4e42c5822c7fc6abef77aa', 'request_id': 'req-7df77ab9-93bf-404e-b302-f48d98a9c6d4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:03.500028 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7df77ab9-93bf-404e-b302-f48d98a9c6d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d6cff138b4e42c5822c7fc6abef77aa', 'request_id': 'req-7df77ab9-93bf-404e-b302-f48d98a9c6d4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:03.506935 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7df77ab9-93bf-404e-b302-f48d98a9c6d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d6cff138b4e42c5822c7fc6abef77aa', 'request_id': 'req-7df77ab9-93bf-404e-b302-f48d98a9c6d4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:03.518895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7df77ab9-93bf-404e-b302-f48d98a9c6d4 admin admin] The token cache is being invalidate because project 6d6cff138b4e42c5822c7fc6abef77aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:55:03.519245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7df77ab9-93bf-404e-b302-f48d98a9c6d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:03.519925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7df77ab9-93bf-404e-b302-f48d98a9c6d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:03.521417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1325/2637] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:55:03 2022] DELETE /identity/v3/projects/6d6cff138b4e42c5822c7fc6abef77aa => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:03.526332 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb174c-7c99-4799-8208-cf6e496935d1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:03.526818 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb174c-7c99-4799-8208-cf6e496935d1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:03.526941 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb174c-7c99-4799-8208-cf6e496935d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:03.547865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d6bb174c-7c99-4799-8208-cf6e496935d1 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:03.549101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d6bb174c-7c99-4799-8208-cf6e496935d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:03.605961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:03.615724 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1313/2638] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:03.624933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6b42f-fa65-4f6b-8cd9-975acb0e9635 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:03.626470 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6b42f-fa65-4f6b-8cd9-975acb0e9635 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:03.627702 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6b42f-fa65-4f6b-8cd9-975acb0e9635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:03.656974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-18b6b42f-fa65-4f6b-8cd9-975acb0e9635 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:03.658304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-18b6b42f-fa65-4f6b-8cd9-975acb0e9635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:03.666561 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1326/2639] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:03.671549 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb174c-7c99-4799-8208-cf6e496935d1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:03.681939 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb174c-7c99-4799-8208-cf6e496935d1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:03.688179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337e45e-ed77-41fd-8476-c8df84c08270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:03.688179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e45e-ed77-41fd-8476-c8df84c08270 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:03.688786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e45e-ed77-41fd-8476-c8df84c08270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:03.688786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e45e-ed77-41fd-8476-c8df84c08270 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:03.689489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337e45e-ed77-41fd-8476-c8df84c08270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:55:03.689844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337e45e-ed77-41fd-8476-c8df84c08270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:03.691502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337e45e-ed77-41fd-8476-c8df84c08270 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:03.705095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1314/2640] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:55:03.709779 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6b42f-fa65-4f6b-8cd9-975acb0e9635 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:03.720458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6b42f-fa65-4f6b-8cd9-975acb0e9635 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:03.725415 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89257d74-3cdd-49c1-ab5b-0dd867dd7c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:03.726026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89257d74-3cdd-49c1-ab5b-0dd867dd7c4c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:03.726338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89257d74-3cdd-49c1-ab5b-0dd867dd7c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:03.726602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89257d74-3cdd-49c1-ab5b-0dd867dd7c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:03.727255 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89257d74-3cdd-49c1-ab5b-0dd867dd7c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:03.729172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89257d74-3cdd-49c1-ab5b-0dd867dd7c4c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:03.740880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:03.827188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1327/2641] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:55:03 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:55:03.836987 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337e45e-ed77-41fd-8476-c8df84c08270 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:03.842278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337e45e-ed77-41fd-8476-c8df84c08270 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:03.923386 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e86fc-f5ea-4958-aad3-2cfda8d4c422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:03.924000 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e86fc-f5ea-4958-aad3-2cfda8d4c422 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:03.924276 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e86fc-f5ea-4958-aad3-2cfda8d4c422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:03.924562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e86fc-f5ea-4958-aad3-2cfda8d4c422 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:03.925195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e86fc-f5ea-4958-aad3-2cfda8d4c422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:03.927376 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e86fc-f5ea-4958-aad3-2cfda8d4c422 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:04.049793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1315/2642] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:55:03 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:55:04.054240 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89257d74-3cdd-49c1-ab5b-0dd867dd7c4c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:04.068056 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89257d74-3cdd-49c1-ab5b-0dd867dd7c4c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:04.075611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5dee2-4f69-4ef0-879d-9bd161b2ffd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:04.076397 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5dee2-4f69-4ef0-879d-9bd161b2ffd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:04.076740 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5dee2-4f69-4ef0-879d-9bd161b2ffd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:04.077071 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5dee2-4f69-4ef0-879d-9bd161b2ffd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:04.081853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5dee2-4f69-4ef0-879d-9bd161b2ffd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:04.081853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5dee2-4f69-4ef0-879d-9bd161b2ffd5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:04.094976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1328/2643] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:04.100787 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e86fc-f5ea-4958-aad3-2cfda8d4c422 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:04.119314 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e86fc-f5ea-4958-aad3-2cfda8d4c422 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:04.128749 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc9053-4b15-4f54-b1d7-a837e74280e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:04.129887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc9053-4b15-4f54-b1d7-a837e74280e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:04.130439 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc9053-4b15-4f54-b1d7-a837e74280e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:04.130907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc9053-4b15-4f54-b1d7-a837e74280e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:04.132233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc9053-4b15-4f54-b1d7-a837e74280e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:04.136793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc9053-4b15-4f54-b1d7-a837e74280e3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:04.160684 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1316/2644] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:04.167036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5dee2-4f69-4ef0-879d-9bd161b2ffd5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:04.189529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5dee2-4f69-4ef0-879d-9bd161b2ffd5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:04.197903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf4d99-de29-4b84-b0fe-96e55cc66f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:04.199107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf4d99-de29-4b84-b0fe-96e55cc66f0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:04.199542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf4d99-de29-4b84-b0fe-96e55cc66f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:04.199997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf4d99-de29-4b84-b0fe-96e55cc66f0c admin admin] PATH_INFO: `/v3/projects/4380d78f1ab44b31a248e882cee43e0e/users/f21526b073494ff2a5f16f4776897ddf/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:04.241185 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf4d99-de29-4b84-b0fe-96e55cc66f0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4380d78f1ab44b31a248e882cee43e0e, user_id=f21526b073494ff2a5f16f4776897ddf, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:04.256526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf4d99-de29-4b84-b0fe-96e55cc66f0c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:04.345508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1329/2645] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:04 2022] PUT /identity/v3/projects/4380d78f1ab44b31a248e882cee43e0e/users/f21526b073494ff2a5f16f4776897ddf/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:04.357031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e9573-fbf2-4c2f-aaba-ca9fe79bc582 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:04.357312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e9573-fbf2-4c2f-aaba-ca9fe79bc582 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:04.357579 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e9573-fbf2-4c2f-aaba-ca9fe79bc582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:04.393043 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c74e9573-fbf2-4c2f-aaba-ca9fe79bc582 None None] MFA Rules not processed for user `f21526b073494ff2a5f16f4776897ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:04.394324 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c74e9573-fbf2-4c2f-aaba-ca9fe79bc582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:04.480524 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1317/2646] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:55:04.487015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf4d99-de29-4b84-b0fe-96e55cc66f0c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:04.573117 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf4d99-de29-4b84-b0fe-96e55cc66f0c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:04.580840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84922505-0095-4a8e-9647-e75102e18155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:04.581841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84922505-0095-4a8e-9647-e75102e18155 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:04.582298 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84922505-0095-4a8e-9647-e75102e18155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:04.582928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84922505-0095-4a8e-9647-e75102e18155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:04.584079 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84922505-0095-4a8e-9647-e75102e18155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:04.587676 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84922505-0095-4a8e-9647-e75102e18155 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:04.704913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1330/2647] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:55:04 2022] POST /identity/v3/projects => generated 385 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:55:04.706285 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e9573-fbf2-4c2f-aaba-ca9fe79bc582 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:04.729224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e9573-fbf2-4c2f-aaba-ca9fe79bc582 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:04.794755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:04.803078 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bb403-2004-4ce0-9cb9-c6adf71b1bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:04.803696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bb403-2004-4ce0-9cb9-c6adf71b1bbb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:04.803975 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bb403-2004-4ce0-9cb9-c6adf71b1bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:04.804247 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bb403-2004-4ce0-9cb9-c6adf71b1bbb admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:04.804887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bb403-2004-4ce0-9cb9-c6adf71b1bbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:04.806960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bb403-2004-4ce0-9cb9-c6adf71b1bbb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:04.912608 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1318/2648] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:55:04 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:55:04.918807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84922505-0095-4a8e-9647-e75102e18155 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:04.933364 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84922505-0095-4a8e-9647-e75102e18155 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:04.940797 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae44a6-0fa6-42b3-ba04-daeba9ac3ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:04.941534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae44a6-0fa6-42b3-ba04-daeba9ac3ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:04.941821 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae44a6-0fa6-42b3-ba04-daeba9ac3ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:04.942087 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae44a6-0fa6-42b3-ba04-daeba9ac3ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:04.942783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae44a6-0fa6-42b3-ba04-daeba9ac3ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:04.944452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae44a6-0fa6-42b3-ba04-daeba9ac3ef5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:04.960982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1331/2649] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:55:04.965243 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bb403-2004-4ce0-9cb9-c6adf71b1bbb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:04.984551 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bb403-2004-4ce0-9cb9-c6adf71b1bbb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:04.998339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfec0d3-9028-4088-beb4-9f62f48a4f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:04.999817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfec0d3-9028-4088-beb4-9f62f48a4f87 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:05.000320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfec0d3-9028-4088-beb4-9f62f48a4f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:05.000817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfec0d3-9028-4088-beb4-9f62f48a4f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:05.002225 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfec0d3-9028-4088-beb4-9f62f48a4f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:05.005335 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfec0d3-9028-4088-beb4-9f62f48a4f87 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:05.024521 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1319/2650] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:05.028773 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae44a6-0fa6-42b3-ba04-daeba9ac3ef5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:05.042269 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae44a6-0fa6-42b3-ba04-daeba9ac3ef5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:05.049374 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5a38d-e892-4559-9cc5-595a842254ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:05.050047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5a38d-e892-4559-9cc5-595a842254ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:05.050388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5a38d-e892-4559-9cc5-595a842254ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:05.050655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5a38d-e892-4559-9cc5-595a842254ec admin admin] PATH_INFO: `/v3/projects/048da19aa5b54b3fa3b356bab5d16bfb/users/4ba996ccf83841eba04e901b1beed32c/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:05.066761 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5a38d-e892-4559-9cc5-595a842254ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=048da19aa5b54b3fa3b356bab5d16bfb, user_id=4ba996ccf83841eba04e901b1beed32c, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:05.069516 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5a38d-e892-4559-9cc5-595a842254ec admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:05.151712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1332/2651] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:05 2022] PUT /identity/v3/projects/048da19aa5b54b3fa3b356bab5d16bfb/users/4ba996ccf83841eba04e901b1beed32c/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:05.164425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab7665-9565-4ee9-8411-d8061e0558fe None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:05.164818 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab7665-9565-4ee9-8411-d8061e0558fe None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:05.165228 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab7665-9565-4ee9-8411-d8061e0558fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:05.198753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f3ab7665-9565-4ee9-8411-d8061e0558fe None None] MFA Rules not processed for user `4ba996ccf83841eba04e901b1beed32c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:05.200033 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f3ab7665-9565-4ee9-8411-d8061e0558fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:05.276720 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1320/2652] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:05.282328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5a38d-e892-4559-9cc5-595a842254ec admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:05.348729 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5a38d-e892-4559-9cc5-595a842254ec admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:05.353473 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a28e4-4533-4780-b1d4-90b3cc37dc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:05.354213 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a28e4-4533-4780-b1d4-90b3cc37dc34 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:05.354483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a28e4-4533-4780-b1d4-90b3cc37dc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:05.354758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a28e4-4533-4780-b1d4-90b3cc37dc34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:05.355397 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a28e4-4533-4780-b1d4-90b3cc37dc34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:05.357301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a28e4-4533-4780-b1d4-90b3cc37dc34 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:05.368006 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:05.546738 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1333/2653] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:55:05 2022] POST /identity/v3/projects => generated 383 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:55:05.550647 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab7665-9565-4ee9-8411-d8061e0558fe None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:05.563115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab7665-9565-4ee9-8411-d8061e0558fe None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:05.640573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f65d98-43af-414a-8a40-3bb162e167c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:05.641608 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f65d98-43af-414a-8a40-3bb162e167c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:05.642087 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f65d98-43af-414a-8a40-3bb162e167c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:05.642622 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f65d98-43af-414a-8a40-3bb162e167c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:05.643768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f65d98-43af-414a-8a40-3bb162e167c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:05.647379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f65d98-43af-414a-8a40-3bb162e167c5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:05.768460 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1321/2654] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:55:05 2022] POST /identity/v3/users => generated 298 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:55:05.772850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a28e4-4533-4780-b1d4-90b3cc37dc34 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:05.784071 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a28e4-4533-4780-b1d4-90b3cc37dc34 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:05.788515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d6a37-f021-444e-a2d2-5d1b6189e4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:05.789168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d6a37-f021-444e-a2d2-5d1b6189e4af admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:05.789495 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d6a37-f021-444e-a2d2-5d1b6189e4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:05.789655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d6a37-f021-444e-a2d2-5d1b6189e4af admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:05.790325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d6a37-f021-444e-a2d2-5d1b6189e4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:05.791942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d6a37-f021-444e-a2d2-5d1b6189e4af admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:05.802119 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1334/2655] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:05.806446 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f65d98-43af-414a-8a40-3bb162e167c5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:05.817491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f65d98-43af-414a-8a40-3bb162e167c5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:05.823168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed0814-5c84-4979-be31-709802eaa0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:05.823824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed0814-5c84-4979-be31-709802eaa0b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:05.824100 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed0814-5c84-4979-be31-709802eaa0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:05.824379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed0814-5c84-4979-be31-709802eaa0b7 admin admin] PATH_INFO: `/v3/projects/609a78743c594d82a1705d31b4358d42/users/9a6066d275a747e0955fe14964434036/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:05.840347 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed0814-5c84-4979-be31-709802eaa0b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609a78743c594d82a1705d31b4358d42, user_id=9a6066d275a747e0955fe14964434036, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:05.843121 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed0814-5c84-4979-be31-709802eaa0b7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:05.981104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1322/2656] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:05 2022] PUT /identity/v3/projects/609a78743c594d82a1705d31b4358d42/users/9a6066d275a747e0955fe14964434036/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:05.985871 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d6a37-f021-444e-a2d2-5d1b6189e4af admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:05.996813 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d6a37-f021-444e-a2d2-5d1b6189e4af admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:06.055888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd78707-bf87-476e-965e-06a2d02e1d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:06.056549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd78707-bf87-476e-965e-06a2d02e1d82 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:06.056964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd78707-bf87-476e-965e-06a2d02e1d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:06.057234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd78707-bf87-476e-965e-06a2d02e1d82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:06.057980 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd78707-bf87-476e-965e-06a2d02e1d82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:06.059654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd78707-bf87-476e-965e-06a2d02e1d82 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:06.072000 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1335/2657] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:06.077045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed0814-5c84-4979-be31-709802eaa0b7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:06.090122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed0814-5c84-4979-be31-709802eaa0b7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:06.094722 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be467f53-d614-4d9b-8c4e-b7225767a521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:06.095398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be467f53-d614-4d9b-8c4e-b7225767a521 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:06.098189 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be467f53-d614-4d9b-8c4e-b7225767a521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:06.098189 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be467f53-d614-4d9b-8c4e-b7225767a521 admin admin] PATH_INFO: `/v3/projects/609a78743c594d82a1705d31b4358d42/users/9a6066d275a747e0955fe14964434036/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:06.098189 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be467f53-d614-4d9b-8c4e-b7225767a521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609a78743c594d82a1705d31b4358d42, user_id=9a6066d275a747e0955fe14964434036, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:06.100648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be467f53-d614-4d9b-8c4e-b7225767a521 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:06.113841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1323/2658] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:06 2022] PUT /identity/v3/projects/609a78743c594d82a1705d31b4358d42/users/9a6066d275a747e0955fe14964434036/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:06.125726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b97a5-c271-44b5-9f8a-4dd5f1aa68c7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:06.126851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b97a5-c271-44b5-9f8a-4dd5f1aa68c7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:06.126851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b97a5-c271-44b5-9f8a-4dd5f1aa68c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:06.158358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-764b97a5-c271-44b5-9f8a-4dd5f1aa68c7 None None] MFA Rules not processed for user `9a6066d275a747e0955fe14964434036`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:06.159625 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-764b97a5-c271-44b5-9f8a-4dd5f1aa68c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:06.261469 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1336/2659] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:06.273891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be467f53-d614-4d9b-8c4e-b7225767a521 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:06.275971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-be467f53-d614-4d9b-8c4e-b7225767a521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:06.300796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:06.349219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be467f53-d614-4d9b-8c4e-b7225767a521 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:06.352890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea054e20-c4fa-4992-8b58-a11d34f70c41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:06.353511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea054e20-c4fa-4992-8b58-a11d34f70c41 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:06.353792 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea054e20-c4fa-4992-8b58-a11d34f70c41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:06.354059 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea054e20-c4fa-4992-8b58-a11d34f70c41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:06.415447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea054e20-c4fa-4992-8b58-a11d34f70c41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:06.416897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea054e20-c4fa-4992-8b58-a11d34f70c41 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:06.427900 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1324/2660] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:55:08.768088 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b97a5-c271-44b5-9f8a-4dd5f1aa68c7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:08.769663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-764b97a5-c271-44b5-9f8a-4dd5f1aa68c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:08.807109 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b97a5-c271-44b5-9f8a-4dd5f1aa68c7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:08.810687 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1b27f-634d-4818-85b7-bd8c6d99454a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:08.811285 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1b27f-634d-4818-85b7-bd8c6d99454a service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:08.812134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1b27f-634d-4818-85b7-bd8c6d99454a service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:08.812400 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1b27f-634d-4818-85b7-bd8c6d99454a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:08.863127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:08.881612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1b27f-634d-4818-85b7-bd8c6d99454a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:08.883073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1b27f-634d-4818-85b7-bd8c6d99454a service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:08.892590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1337/2661] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:55:10.076657 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea054e20-c4fa-4992-8b58-a11d34f70c41 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:10.079218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ea054e20-c4fa-4992-8b58-a11d34f70c41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:10.147584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea054e20-c4fa-4992-8b58-a11d34f70c41 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:10.153348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b22f60-016e-41e4-8470-ded6fc604273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:10.153970 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b22f60-016e-41e4-8470-ded6fc604273 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:10.154280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b22f60-016e-41e4-8470-ded6fc604273 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:10.154554 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b22f60-016e-41e4-8470-ded6fc604273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:10.157018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d2b22f60-016e-41e4-8470-ded6fc604273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:10.231903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b22f60-016e-41e4-8470-ded6fc604273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:10.233431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b22f60-016e-41e4-8470-ded6fc604273 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:10.245195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1325/2662] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:55:10.774140 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1b27f-634d-4818-85b7-bd8c6d99454a service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:10.786992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1b27f-634d-4818-85b7-bd8c6d99454a service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:10.793343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3234cee-1bf3-43e0-94e7-b2cbcfb5cac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:10.794388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3234cee-1bf3-43e0-94e7-b2cbcfb5cac3 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:10.794864 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3234cee-1bf3-43e0-94e7-b2cbcfb5cac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:10.795194 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3234cee-1bf3-43e0-94e7-b2cbcfb5cac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:10.808616 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3234cee-1bf3-43e0-94e7-b2cbcfb5cac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:10.812035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3234cee-1bf3-43e0-94e7-b2cbcfb5cac3 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:10.831065 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1338/2663] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:55:14.415170 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b22f60-016e-41e4-8470-ded6fc604273 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:14.416766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d2b22f60-016e-41e4-8470-ded6fc604273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:14.429936 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b22f60-016e-41e4-8470-ded6fc604273 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:14.435141 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bac10-22e0-4a42-8049-d1382030a576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:14.436259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bac10-22e0-4a42-8049-d1382030a576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:14.436259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bac10-22e0-4a42-8049-d1382030a576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:14.436259 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bac10-22e0-4a42-8049-d1382030a576 admin admin] PATH_INFO: `/v3/users/64ea6fbc6e7d42dabdf459e8de4c3621` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:14.437444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bac10-22e0-4a42-8049-d1382030a576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ea6fbc6e7d42dabdf459e8de4c3621)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:14.439430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bac10-22e0-4a42-8049-d1382030a576 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:14.458555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:14.661875 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-292bac10-22e0-4a42-8049-d1382030a576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ea6fbc6e7d42dabdf459e8de4c3621'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:14.681092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-292bac10-22e0-4a42-8049-d1382030a576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ea6fbc6e7d42dabdf459e8de4c3621'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:14.769021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-292bac10-22e0-4a42-8049-d1382030a576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ea6fbc6e7d42dabdf459e8de4c3621'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:14.769655 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-292bac10-22e0-4a42-8049-d1382030a576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ea6fbc6e7d42dabdf459e8de4c3621'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:14.777672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-292bac10-22e0-4a42-8049-d1382030a576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ea6fbc6e7d42dabdf459e8de4c3621'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:14.784411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1326/2664] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:55:14 2022] DELETE /identity/v3/users/64ea6fbc6e7d42dabdf459e8de4c3621 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:15.033934 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3234cee-1bf3-43e0-94e7-b2cbcfb5cac3 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:15.045305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3234cee-1bf3-43e0-94e7-b2cbcfb5cac3 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:15.117617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:15.118286 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:15.118612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:15.118833 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b admin admin] PATH_INFO: `/v3/projects/018b87eeeda248e4a85f587c31e297f4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:15.120064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b admin admin] RBAC: Authorizing `identity:delete_project(project_id=018b87eeeda248e4a85f587c31e297f4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:15.122004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:15.225456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '018b87eeeda248e4a85f587c31e297f4', 'request_id': 'req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:15.299507 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '018b87eeeda248e4a85f587c31e297f4', 'request_id': 'req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:15.306297 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '018b87eeeda248e4a85f587c31e297f4', 'request_id': 'req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:15.318673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b admin admin] The token cache is being invalidate because project 018b87eeeda248e4a85f587c31e297f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:55:15.318960 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:15.319611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8fbd1a7-72ba-4ec4-9d50-574338eca54b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:15.321101 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1339/2665] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:55:15 2022] DELETE /identity/v3/projects/018b87eeeda248e4a85f587c31e297f4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:15.326138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8c4da-e90e-4d72-aee9-0cb5c1475b02 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:15.326449 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8c4da-e90e-4d72-aee9-0cb5c1475b02 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:15.327231 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8c4da-e90e-4d72-aee9-0cb5c1475b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:15.349079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-52c8c4da-e90e-4d72-aee9-0cb5c1475b02 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:15.350370 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-52c8c4da-e90e-4d72-aee9-0cb5c1475b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:15.415166 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1327/2666] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:15.423090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c1353-5b4e-4ac5-b1ea-c1459d425828 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:15.423401 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c1353-5b4e-4ac5-b1ea-c1459d425828 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:15.423672 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c1353-5b4e-4ac5-b1ea-c1459d425828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:15.443583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-509c1353-5b4e-4ac5-b1ea-c1459d425828 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:15.444873 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-509c1353-5b4e-4ac5-b1ea-c1459d425828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:15.456857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1340/2667] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:15.458481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8c4da-e90e-4d72-aee9-0cb5c1475b02 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:15.468964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8c4da-e90e-4d72-aee9-0cb5c1475b02 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:15.473562 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f18ec-4553-4921-8bc1-765a975b947a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:15.474168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f18ec-4553-4921-8bc1-765a975b947a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:15.474455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f18ec-4553-4921-8bc1-765a975b947a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:15.474743 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f18ec-4553-4921-8bc1-765a975b947a admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:15.475428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f18ec-4553-4921-8bc1-765a975b947a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:55:15.475690 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f18ec-4553-4921-8bc1-765a975b947a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:15.477291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f18ec-4553-4921-8bc1-765a975b947a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:15.489525 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1328/2668] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:55:15.494037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c1353-5b4e-4ac5-b1ea-c1459d425828 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:15.504305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c1353-5b4e-4ac5-b1ea-c1459d425828 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:15.509183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ee1a2-4b8d-4511-ba53-51ec7c1967af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:15.509865 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ee1a2-4b8d-4511-ba53-51ec7c1967af admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:15.510135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ee1a2-4b8d-4511-ba53-51ec7c1967af admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:15.510423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ee1a2-4b8d-4511-ba53-51ec7c1967af admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:15.511058 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ee1a2-4b8d-4511-ba53-51ec7c1967af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:15.512938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ee1a2-4b8d-4511-ba53-51ec7c1967af admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:15.606118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1341/2669] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:55:15 2022] POST /identity/v3/projects => generated 357 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:55:15.611025 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f18ec-4553-4921-8bc1-765a975b947a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:15.622185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f18ec-4553-4921-8bc1-765a975b947a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:15.683013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a44e91-c0be-428c-82b1-e0a97e353d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:15.683668 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a44e91-c0be-428c-82b1-e0a97e353d5f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:15.683945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a44e91-c0be-428c-82b1-e0a97e353d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:15.684219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a44e91-c0be-428c-82b1-e0a97e353d5f admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:15.684885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a44e91-c0be-428c-82b1-e0a97e353d5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:15.687366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a44e91-c0be-428c-82b1-e0a97e353d5f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:15.777274 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1329/2670] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:55:15 2022] POST /identity/v3/users => generated 279 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:55:15.778393 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ee1a2-4b8d-4511-ba53-51ec7c1967af admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:15.788758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ee1a2-4b8d-4511-ba53-51ec7c1967af admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:15.794204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac15c41-6952-4a63-b14e-2384493afbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:15.794805 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac15c41-6952-4a63-b14e-2384493afbdc admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:15.795074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac15c41-6952-4a63-b14e-2384493afbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:15.795337 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac15c41-6952-4a63-b14e-2384493afbdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:15.795971 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac15c41-6952-4a63-b14e-2384493afbdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:15.797534 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac15c41-6952-4a63-b14e-2384493afbdc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:15.812044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1342/2671] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:15.813336 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a44e91-c0be-428c-82b1-e0a97e353d5f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:15.820827 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac15c41-6952-4a63-b14e-2384493afbdc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:15.823089 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8ac15c41-6952-4a63-b14e-2384493afbdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:15.830240 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a44e91-c0be-428c-82b1-e0a97e353d5f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:15.835160 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4473b9-6e6d-4ef5-9bcf-692120583841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:15.835759 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4473b9-6e6d-4ef5-9bcf-692120583841 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:15.836038 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4473b9-6e6d-4ef5-9bcf-692120583841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:15.836153 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac15c41-6952-4a63-b14e-2384493afbdc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:15.836310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4473b9-6e6d-4ef5-9bcf-692120583841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:15.837013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4473b9-6e6d-4ef5-9bcf-692120583841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:15.838559 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4473b9-6e6d-4ef5-9bcf-692120583841 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:15.840558 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7d115-5223-4ba1-8d40-2e3bd8d596ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:15.840980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:15.841169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7d115-5223-4ba1-8d40-2e3bd8d596ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:15.841441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7d115-5223-4ba1-8d40-2e3bd8d596ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:15.841705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7d115-5223-4ba1-8d40-2e3bd8d596ea admin admin] PATH_INFO: `/v3/users/f21526b073494ff2a5f16f4776897ddf` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:15.842860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7d115-5223-4ba1-8d40-2e3bd8d596ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=f21526b073494ff2a5f16f4776897ddf)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:15.844853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7d115-5223-4ba1-8d40-2e3bd8d596ea admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:15.850063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1330/2672] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:15.854255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4473b9-6e6d-4ef5-9bcf-692120583841 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:15.856850 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:15.869137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4473b9-6e6d-4ef5-9bcf-692120583841 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:15.875411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfd2da-1344-4987-8e88-30ae0fcd23cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:15.876217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfd2da-1344-4987-8e88-30ae0fcd23cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:15.876942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfd2da-1344-4987-8e88-30ae0fcd23cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:15.877225 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfd2da-1344-4987-8e88-30ae0fcd23cc admin admin] PATH_INFO: `/v3/projects/1d249b7417674eed8d0357bb300c07ec/users/eedbf5da0a1f4cf387497f0f58f0d214/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:15.893338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfd2da-1344-4987-8e88-30ae0fcd23cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d249b7417674eed8d0357bb300c07ec, user_id=eedbf5da0a1f4cf387497f0f58f0d214, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:15.896035 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfd2da-1344-4987-8e88-30ae0fcd23cc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:15.962456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1331/2673] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:15 2022] PUT /identity/v3/projects/1d249b7417674eed8d0357bb300c07ec/users/eedbf5da0a1f4cf387497f0f58f0d214/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:55:15.974698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f87dab-e728-45dc-a5ae-caa2157c722d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:15.974995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f87dab-e728-45dc-a5ae-caa2157c722d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:15.975265 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f87dab-e728-45dc-a5ae-caa2157c722d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:16.009437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-27f87dab-e728-45dc-a5ae-caa2157c722d None None] MFA Rules not processed for user `eedbf5da0a1f4cf387497f0f58f0d214`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:16.010672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-27f87dab-e728-45dc-a5ae-caa2157c722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:16.044268 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61c7d115-5223-4ba1-8d40-2e3bd8d596ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f21526b073494ff2a5f16f4776897ddf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:16.046964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61c7d115-5223-4ba1-8d40-2e3bd8d596ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f21526b073494ff2a5f16f4776897ddf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:16.083465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1332/2674] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:55:16.088269 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f87dab-e728-45dc-a5ae-caa2157c722d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:16.089703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-27f87dab-e728-45dc-a5ae-caa2157c722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:16.140430 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61c7d115-5223-4ba1-8d40-2e3bd8d596ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f21526b073494ff2a5f16f4776897ddf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:16.148293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61c7d115-5223-4ba1-8d40-2e3bd8d596ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f21526b073494ff2a5f16f4776897ddf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:16.161707 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61c7d115-5223-4ba1-8d40-2e3bd8d596ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f21526b073494ff2a5f16f4776897ddf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:16.162095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f87dab-e728-45dc-a5ae-caa2157c722d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:16.166873 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b3e62-c9ee-4094-b67c-ce408332b639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:16.167476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b3e62-c9ee-4094-b67c-ce408332b639 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:16.167747 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b3e62-c9ee-4094-b67c-ce408332b639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:16.168036 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b3e62-c9ee-4094-b67c-ce408332b639 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:16.168668 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b3e62-c9ee-4094-b67c-ce408332b639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:16.168785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1343/2675] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:55:15 2022] DELETE /identity/v3/users/f21526b073494ff2a5f16f4776897ddf => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:55:16.170631 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b3e62-c9ee-4094-b67c-ce408332b639 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:16.173071 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbc950-aa9d-47e4-89df-5a99a434f5ba None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:16.173392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbc950-aa9d-47e4-89df-5a99a434f5ba None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:16.173665 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbc950-aa9d-47e4-89df-5a99a434f5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:16.195115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-addbc950-aa9d-47e4-89df-5a99a434f5ba None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:16.196324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-addbc950-aa9d-47e4-89df-5a99a434f5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:16.263974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1344/2676] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:55:16.275664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbc950-aa9d-47e4-89df-5a99a434f5ba None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:16.277116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-addbc950-aa9d-47e4-89df-5a99a434f5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:16.299118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1333/2677] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:55:16 2022] POST /identity/v3/projects => generated 357 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:55:16.304807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b3e62-c9ee-4094-b67c-ce408332b639 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:16.345858 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbc950-aa9d-47e4-89df-5a99a434f5ba None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:16.369395 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:16.386374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b3e62-c9ee-4094-b67c-ce408332b639 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:16.392476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd2cbc-70f2-4589-aed0-4cc50b1d4efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:16.393110 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd2cbc-70f2-4589-aed0-4cc50b1d4efb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:16.393386 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd2cbc-70f2-4589-aed0-4cc50b1d4efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:16.393665 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd2cbc-70f2-4589-aed0-4cc50b1d4efb admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:16.394565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd2cbc-70f2-4589-aed0-4cc50b1d4efb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:16.398620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd2cbc-70f2-4589-aed0-4cc50b1d4efb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:16.402075 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:16.411106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209cd10-b700-4b33-b914-0e260874be79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:16.411734 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209cd10-b700-4b33-b914-0e260874be79 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:16.412007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209cd10-b700-4b33-b914-0e260874be79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:16.412282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209cd10-b700-4b33-b914-0e260874be79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:16.422932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209cd10-b700-4b33-b914-0e260874be79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:16.424458 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209cd10-b700-4b33-b914-0e260874be79 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:16.435712 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1345/2678] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:55:16.498195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1334/2679] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:55:16 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:55:16.502450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209cd10-b700-4b33-b914-0e260874be79 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:16.516361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209cd10-b700-4b33-b914-0e260874be79 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:16.521132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddda4d-848f-4206-a4b7-c62dfc655c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:16.522425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddda4d-848f-4206-a4b7-c62dfc655c71 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:16.522425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddda4d-848f-4206-a4b7-c62dfc655c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:16.522425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddda4d-848f-4206-a4b7-c62dfc655c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:16.522884 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddda4d-848f-4206-a4b7-c62dfc655c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:16.524463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddda4d-848f-4206-a4b7-c62dfc655c71 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:16.535049 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1346/2680] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:55:16.539522 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd2cbc-70f2-4589-aed0-4cc50b1d4efb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:16.551356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd2cbc-70f2-4589-aed0-4cc50b1d4efb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:16.555687 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7680f93-63f8-4172-839e-0acc1e0e7057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:16.556362 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7680f93-63f8-4172-839e-0acc1e0e7057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:16.556635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7680f93-63f8-4172-839e-0acc1e0e7057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:16.556902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7680f93-63f8-4172-839e-0acc1e0e7057 admin admin] PATH_INFO: `/v3/projects/4577fdb978d94d62b4fbc632a18b9f71/users/a233f3d7c447404fba6371dec4b4da50/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:16.572482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7680f93-63f8-4172-839e-0acc1e0e7057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4577fdb978d94d62b4fbc632a18b9f71, user_id=a233f3d7c447404fba6371dec4b4da50, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:16.575147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7680f93-63f8-4172-839e-0acc1e0e7057 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:16.670124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1335/2681] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:16 2022] PUT /identity/v3/projects/4577fdb978d94d62b4fbc632a18b9f71/users/a233f3d7c447404fba6371dec4b4da50/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:55:16.674695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddda4d-848f-4206-a4b7-c62dfc655c71 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:16.685938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddda4d-848f-4206-a4b7-c62dfc655c71 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:16.751908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52065487-564a-4377-86fe-826aeb06b533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:16.752533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52065487-564a-4377-86fe-826aeb06b533 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:16.752814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52065487-564a-4377-86fe-826aeb06b533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:16.753363 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52065487-564a-4377-86fe-826aeb06b533 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:16.753722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52065487-564a-4377-86fe-826aeb06b533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:16.755614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52065487-564a-4377-86fe-826aeb06b533 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:16.767588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1347/2682] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:16.772076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7680f93-63f8-4172-839e-0acc1e0e7057 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:16.782625 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7680f93-63f8-4172-839e-0acc1e0e7057 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:16.787491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf48e5-e2f1-43b1-861a-49c8eb2c39ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:16.788174 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf48e5-e2f1-43b1-861a-49c8eb2c39ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:16.788451 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf48e5-e2f1-43b1-861a-49c8eb2c39ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:16.788719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf48e5-e2f1-43b1-861a-49c8eb2c39ba admin admin] PATH_INFO: `/v3/projects/4577fdb978d94d62b4fbc632a18b9f71/users/a233f3d7c447404fba6371dec4b4da50/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:16.790696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf48e5-e2f1-43b1-861a-49c8eb2c39ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4577fdb978d94d62b4fbc632a18b9f71, user_id=a233f3d7c447404fba6371dec4b4da50, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:16.793338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf48e5-e2f1-43b1-861a-49c8eb2c39ba admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:16.805989 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1336/2683] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:16 2022] PUT /identity/v3/projects/4577fdb978d94d62b4fbc632a18b9f71/users/a233f3d7c447404fba6371dec4b4da50/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:16.817621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba34d6-4ef2-4fc4-92bc-1ac3a91908ef None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:16.817903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba34d6-4ef2-4fc4-92bc-1ac3a91908ef None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:16.818202 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba34d6-4ef2-4fc4-92bc-1ac3a91908ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:16.848596 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-98ba34d6-4ef2-4fc4-92bc-1ac3a91908ef None None] MFA Rules not processed for user `a233f3d7c447404fba6371dec4b4da50`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:16.849710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-98ba34d6-4ef2-4fc4-92bc-1ac3a91908ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:16.931939 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1348/2684] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:16 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:16.951460 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf48e5-e2f1-43b1-861a-49c8eb2c39ba admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:16.951460 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3cbf48e5-e2f1-43b1-861a-49c8eb2c39ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:16.955385 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba34d6-4ef2-4fc4-92bc-1ac3a91908ef None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:16.961701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-98ba34d6-4ef2-4fc4-92bc-1ac3a91908ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:17.033779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf48e5-e2f1-43b1-861a-49c8eb2c39ba admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:17.039224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252aed4-1510-488d-953e-c2169a04282d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:17.039790 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252aed4-1510-488d-953e-c2169a04282d service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:17.040068 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252aed4-1510-488d-953e-c2169a04282d service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:17.040337 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252aed4-1510-488d-953e-c2169a04282d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:17.051509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252aed4-1510-488d-953e-c2169a04282d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:17.052955 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252aed4-1510-488d-953e-c2169a04282d service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:17.055974 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:17.060951 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba34d6-4ef2-4fc4-92bc-1ac3a91908ef None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:17.063940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1337/2685] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:16 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:55:17.067442 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:17.068262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:17.068732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:17.068978 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] PATH_INFO: `/v3/projects/4380d78f1ab44b31a248e882cee43e0e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:17.070226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4380d78f1ab44b31a248e882cee43e0e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:17.072207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:17.239580 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4380d78f1ab44b31a248e882cee43e0e', 'request_id': 'req-a3feceb7-64d3-4706-a321-696ecdf3e984'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:17.337828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4380d78f1ab44b31a248e882cee43e0e', 'request_id': 'req-a3feceb7-64d3-4706-a321-696ecdf3e984'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:17.349948 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4380d78f1ab44b31a248e882cee43e0e', 'request_id': 'req-a3feceb7-64d3-4706-a321-696ecdf3e984'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:17.370600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252aed4-1510-488d-953e-c2169a04282d service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:17.375414 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] The token cache is being invalidate because project 4380d78f1ab44b31a248e882cee43e0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:55:17.375913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:17.376983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:17.378857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1349/2686] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:55:16 2022] DELETE /identity/v3/projects/4380d78f1ab44b31a248e882cee43e0e => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:55:17.382844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:17.383510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252aed4-1510-488d-953e-c2169a04282d service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:17.384612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:17.440407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30699c-4aca-45f8-abed-b5c79e1b7c79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:17.441102 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30699c-4aca-45f8-abed-b5c79e1b7c79 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:17.441384 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30699c-4aca-45f8-abed-b5c79e1b7c79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:17.441636 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30699c-4aca-45f8-abed-b5c79e1b7c79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:17.443544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c30699c-4aca-45f8-abed-b5c79e1b7c79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:17.478453 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3feceb7-64d3-4706-a321-696ecdf3e984 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:17.487989 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:17.489234 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:17.489717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:17.490212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] PATH_INFO: `/v3/users/4ba996ccf83841eba04e901b1beed32c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:17.492154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ba996ccf83841eba04e901b1beed32c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:17.495758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:17.530948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30699c-4aca-45f8-abed-b5c79e1b7c79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:17.533205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30699c-4aca-45f8-abed-b5c79e1b7c79 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:17.538311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:17.546713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1338/2687] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:17 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:55:17.807701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ba996ccf83841eba04e901b1beed32c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:17.808452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ba996ccf83841eba04e901b1beed32c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:17.887851 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ba996ccf83841eba04e901b1beed32c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:17.897404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ba996ccf83841eba04e901b1beed32c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:17.910506 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ba996ccf83841eba04e901b1beed32c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:17.916760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1350/2688] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:55:17 2022] DELETE /identity/v3/users/4ba996ccf83841eba04e901b1beed32c => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:55:18.381367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30699c-4aca-45f8-abed-b5c79e1b7c79 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:18.395462 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30699c-4aca-45f8-abed-b5c79e1b7c79 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:18.468367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:18.469021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:18.469292 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:18.469559 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] PATH_INFO: `/v3/projects/048da19aa5b54b3fa3b356bab5d16bfb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:18.470703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] RBAC: Authorizing `identity:delete_project(project_id=048da19aa5b54b3fa3b356bab5d16bfb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:18.472581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:18.609593 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048da19aa5b54b3fa3b356bab5d16bfb', 'request_id': 'req-475321a6-2056-47fa-ae86-d273e398f26d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:18.693441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048da19aa5b54b3fa3b356bab5d16bfb', 'request_id': 'req-475321a6-2056-47fa-ae86-d273e398f26d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:18.700509 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048da19aa5b54b3fa3b356bab5d16bfb', 'request_id': 'req-475321a6-2056-47fa-ae86-d273e398f26d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:18.713158 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] The token cache is being invalidate because project 048da19aa5b54b3fa3b356bab5d16bfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:55:18.713573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:18.714263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:18.715797 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1339/2689] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:55:18 2022] DELETE /identity/v3/projects/048da19aa5b54b3fa3b356bab5d16bfb => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:18.719992 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:18.724252 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:18.736253 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0404b-e6f0-4dc7-9521-da255a1c9996 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:18.740312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:18.795870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d2c5b-5ebb-4a5d-bb9c-c39f61641ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:18.796948 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d2c5b-5ebb-4a5d-bb9c-c39f61641ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:18.797410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d2c5b-5ebb-4a5d-bb9c-c39f61641ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:18.798391 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d2c5b-5ebb-4a5d-bb9c-c39f61641ac6 admin admin] PATH_INFO: `/v3/users/9a6066d275a747e0955fe14964434036` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:18.799704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d2c5b-5ebb-4a5d-bb9c-c39f61641ac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a6066d275a747e0955fe14964434036)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:18.801696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d2c5b-5ebb-4a5d-bb9c-c39f61641ac6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:19.107812 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b7d2c5b-5ebb-4a5d-bb9c-c39f61641ac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a6066d275a747e0955fe14964434036'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:19.108997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b7d2c5b-5ebb-4a5d-bb9c-c39f61641ac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a6066d275a747e0955fe14964434036'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:19.190487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b7d2c5b-5ebb-4a5d-bb9c-c39f61641ac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a6066d275a747e0955fe14964434036'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:19.199531 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b7d2c5b-5ebb-4a5d-bb9c-c39f61641ac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a6066d275a747e0955fe14964434036'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:19.212035 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b7d2c5b-5ebb-4a5d-bb9c-c39f61641ac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a6066d275a747e0955fe14964434036'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:19.224034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1351/2690] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:55:18 2022] DELETE /identity/v3/users/9a6066d275a747e0955fe14964434036 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:55:19.729369 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:19.730910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:19.747227 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475321a6-2056-47fa-ae86-d273e398f26d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:19.816077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e7eac-e964-4f40-8a22-8346f7ca8c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:19.816705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e7eac-e964-4f40-8a22-8346f7ca8c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:19.816980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e7eac-e964-4f40-8a22-8346f7ca8c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:19.817358 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e7eac-e964-4f40-8a22-8346f7ca8c6d admin admin] PATH_INFO: `/v3/projects/609a78743c594d82a1705d31b4358d42` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:19.818527 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e7eac-e964-4f40-8a22-8346f7ca8c6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=609a78743c594d82a1705d31b4358d42)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:19.820413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e7eac-e964-4f40-8a22-8346f7ca8c6d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:19.922896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:19.936034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-953e7eac-e964-4f40-8a22-8346f7ca8c6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609a78743c594d82a1705d31b4358d42', 'request_id': 'req-953e7eac-e964-4f40-8a22-8346f7ca8c6d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:20.017515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-953e7eac-e964-4f40-8a22-8346f7ca8c6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609a78743c594d82a1705d31b4358d42', 'request_id': 'req-953e7eac-e964-4f40-8a22-8346f7ca8c6d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:20.024398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-953e7eac-e964-4f40-8a22-8346f7ca8c6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609a78743c594d82a1705d31b4358d42', 'request_id': 'req-953e7eac-e964-4f40-8a22-8346f7ca8c6d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:20.037138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-953e7eac-e964-4f40-8a22-8346f7ca8c6d admin admin] The token cache is being invalidate because project 609a78743c594d82a1705d31b4358d42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:55:20.037428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-953e7eac-e964-4f40-8a22-8346f7ca8c6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:20.038098 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-953e7eac-e964-4f40-8a22-8346f7ca8c6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:20.039649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1340/2691] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:55:19 2022] DELETE /identity/v3/projects/609a78743c594d82a1705d31b4358d42 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:20.044615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14a9f6-60a9-4cd1-a5fe-bb14180d43a6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:20.049026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14a9f6-60a9-4cd1-a5fe-bb14180d43a6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:20.049026 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14a9f6-60a9-4cd1-a5fe-bb14180d43a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:20.072681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1b14a9f6-60a9-4cd1-a5fe-bb14180d43a6 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:20.073927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1b14a9f6-60a9-4cd1-a5fe-bb14180d43a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:20.140292 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1352/2692] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:55:20.208453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4cf0c-36fe-44e0-97b2-a484c5da27f3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:20.208745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4cf0c-36fe-44e0-97b2-a484c5da27f3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:20.208998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4cf0c-36fe-44e0-97b2-a484c5da27f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:20.230926 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fed4cf0c-36fe-44e0-97b2-a484c5da27f3 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:20.232200 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fed4cf0c-36fe-44e0-97b2-a484c5da27f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:20.244453 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1341/2693] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:55:20.249647 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14a9f6-60a9-4cd1-a5fe-bb14180d43a6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:20.259951 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14a9f6-60a9-4cd1-a5fe-bb14180d43a6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:20.265215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2885b92-fffc-4ccb-88f5-1f6021f288b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:20.265876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2885b92-fffc-4ccb-88f5-1f6021f288b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:20.266260 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2885b92-fffc-4ccb-88f5-1f6021f288b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:20.266588 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2885b92-fffc-4ccb-88f5-1f6021f288b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:20.267043 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2885b92-fffc-4ccb-88f5-1f6021f288b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:55:20.267423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2885b92-fffc-4ccb-88f5-1f6021f288b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:20.269116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2885b92-fffc-4ccb-88f5-1f6021f288b5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:20.282053 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1353/2694] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:55:20.286740 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4cf0c-36fe-44e0-97b2-a484c5da27f3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:20.297001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4cf0c-36fe-44e0-97b2-a484c5da27f3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:20.301984 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e9937-2675-4e25-bd3a-ff73a68740e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:20.302644 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e9937-2675-4e25-bd3a-ff73a68740e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:20.302923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e9937-2675-4e25-bd3a-ff73a68740e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:20.303199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e9937-2675-4e25-bd3a-ff73a68740e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:20.303937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e9937-2675-4e25-bd3a-ff73a68740e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:20.305887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e9937-2675-4e25-bd3a-ff73a68740e8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:20.403648 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1342/2695] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:55:20 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:55:20.408688 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2885b92-fffc-4ccb-88f5-1f6021f288b5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:20.419747 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2885b92-fffc-4ccb-88f5-1f6021f288b5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:20.430337 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:20.483725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6aa82b-7f97-454e-9376-d6412ef68264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:20.484356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6aa82b-7f97-454e-9376-d6412ef68264 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:20.484641 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6aa82b-7f97-454e-9376-d6412ef68264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:20.484909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6aa82b-7f97-454e-9376-d6412ef68264 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:20.485814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6aa82b-7f97-454e-9376-d6412ef68264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:20.487802 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6aa82b-7f97-454e-9376-d6412ef68264 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:20.602886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1354/2696] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:55:20 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:55:20.607063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e9937-2675-4e25-bd3a-ff73a68740e8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:20.619657 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e9937-2675-4e25-bd3a-ff73a68740e8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:20.625157 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e48904-d20d-4253-b15b-fd35e7150c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:20.625767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e48904-d20d-4253-b15b-fd35e7150c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:20.626045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e48904-d20d-4253-b15b-fd35e7150c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:20.626353 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e48904-d20d-4253-b15b-fd35e7150c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:20.627005 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e48904-d20d-4253-b15b-fd35e7150c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:20.628628 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e48904-d20d-4253-b15b-fd35e7150c60 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:20.639825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1343/2697] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:20.644045 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6aa82b-7f97-454e-9376-d6412ef68264 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:20.655994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6aa82b-7f97-454e-9376-d6412ef68264 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:20.660763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8bf873-9f3d-495e-939e-c43f1d1f1e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:20.661358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bf873-9f3d-495e-939e-c43f1d1f1e14 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:20.661628 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bf873-9f3d-495e-939e-c43f1d1f1e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:20.661903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bf873-9f3d-495e-939e-c43f1d1f1e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:20.662566 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8bf873-9f3d-495e-939e-c43f1d1f1e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:20.664293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8bf873-9f3d-495e-939e-c43f1d1f1e14 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:20.677853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1355/2698] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:20.683042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e48904-d20d-4253-b15b-fd35e7150c60 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:20.694434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e48904-d20d-4253-b15b-fd35e7150c60 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:20.699539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cda004-3389-4e92-bc67-9445e25d1d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:20.700463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cda004-3389-4e92-bc67-9445e25d1d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:20.700844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cda004-3389-4e92-bc67-9445e25d1d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:20.701220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cda004-3389-4e92-bc67-9445e25d1d8a admin admin] PATH_INFO: `/v3/projects/628b91e13fdd4830ab5d768b7dd794a5/users/57a6c9d662374991a0dd148e40af864c/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:20.719554 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cda004-3389-4e92-bc67-9445e25d1d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=628b91e13fdd4830ab5d768b7dd794a5, user_id=57a6c9d662374991a0dd148e40af864c, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:20.722417 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cda004-3389-4e92-bc67-9445e25d1d8a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:20.849255 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1344/2699] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:20 2022] PUT /identity/v3/projects/628b91e13fdd4830ab5d768b7dd794a5/users/57a6c9d662374991a0dd148e40af864c/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:20.861518 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc345aa-5513-42f6-a49a-15b5c3bc65f8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:20.861860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc345aa-5513-42f6-a49a-15b5c3bc65f8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:20.862040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc345aa-5513-42f6-a49a-15b5c3bc65f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:20.897263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4dc345aa-5513-42f6-a49a-15b5c3bc65f8 None None] MFA Rules not processed for user `57a6c9d662374991a0dd148e40af864c`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:20.898517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4dc345aa-5513-42f6-a49a-15b5c3bc65f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:20.975959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1356/2700] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:55:20.989185 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cda004-3389-4e92-bc67-9445e25d1d8a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:20.991161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-13cda004-3389-4e92-bc67-9445e25d1d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:21.067136 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cda004-3389-4e92-bc67-9445e25d1d8a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:21.071178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bae599-edcc-4d94-932e-818620ba7e50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:21.071795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bae599-edcc-4d94-932e-818620ba7e50 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:21.072073 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bae599-edcc-4d94-932e-818620ba7e50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:21.072352 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bae599-edcc-4d94-932e-818620ba7e50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:21.083295 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bae599-edcc-4d94-932e-818620ba7e50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:21.086811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bae599-edcc-4d94-932e-818620ba7e50 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:21.097583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1345/2701] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:55:30.972775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=60,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:31.092275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=66,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:42.720663 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc345aa-5513-42f6-a49a-15b5c3bc65f8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:42.722754 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4dc345aa-5513-42f6-a49a-15b5c3bc65f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:42.827357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc345aa-5513-42f6-a49a-15b5c3bc65f8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:42.834037 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e9f95-3984-4a72-bedc-f3d56e170ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:42.834947 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e9f95-3984-4a72-bedc-f3d56e170ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:42.835360 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e9f95-3984-4a72-bedc-f3d56e170ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:42.835727 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e9f95-3984-4a72-bedc-f3d56e170ae6 admin admin] PATH_INFO: `/v3/users/eedbf5da0a1f4cf387497f0f58f0d214` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:42.837206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e9f95-3984-4a72-bedc-f3d56e170ae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eedbf5da0a1f4cf387497f0f58f0d214)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:42.840067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e9f95-3984-4a72-bedc-f3d56e170ae6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:42.861868 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bae599-edcc-4d94-932e-818620ba7e50 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:42.879651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bae599-edcc-4d94-932e-818620ba7e50 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:42.884914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77820d0-3d5b-46ca-aa4c-db445b674149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:42.885796 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77820d0-3d5b-46ca-aa4c-db445b674149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:42.886312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77820d0-3d5b-46ca-aa4c-db445b674149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:42.886582 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77820d0-3d5b-46ca-aa4c-db445b674149 admin admin] PATH_INFO: `/v3/users/57a6c9d662374991a0dd148e40af864c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:42.887817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77820d0-3d5b-46ca-aa4c-db445b674149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57a6c9d662374991a0dd148e40af864c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:42.889788 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77820d0-3d5b-46ca-aa4c-db445b674149 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:43.120714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e3e9f95-3984-4a72-bedc-f3d56e170ae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eedbf5da0a1f4cf387497f0f58f0d214'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:43.121677 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e3e9f95-3984-4a72-bedc-f3d56e170ae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eedbf5da0a1f4cf387497f0f58f0d214'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:43.216094 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b77820d0-3d5b-46ca-aa4c-db445b674149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57a6c9d662374991a0dd148e40af864c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:43.236006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b77820d0-3d5b-46ca-aa4c-db445b674149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57a6c9d662374991a0dd148e40af864c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:43.236207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e3e9f95-3984-4a72-bedc-f3d56e170ae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eedbf5da0a1f4cf387497f0f58f0d214'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:43.248674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e3e9f95-3984-4a72-bedc-f3d56e170ae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eedbf5da0a1f4cf387497f0f58f0d214'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:43.270557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e3e9f95-3984-4a72-bedc-f3d56e170ae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eedbf5da0a1f4cf387497f0f58f0d214'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:43.282331 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1357/2702] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:55:42 2022] DELETE /identity/v3/users/eedbf5da0a1f4cf387497f0f58f0d214 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:55:43.287774 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db558d7-3e09-4ff8-8fd7-b0cfc223de73 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:43.288272 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db558d7-3e09-4ff8-8fd7-b0cfc223de73 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:43.288758 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db558d7-3e09-4ff8-8fd7-b0cfc223de73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:43.320307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9db558d7-3e09-4ff8-8fd7-b0cfc223de73 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:43.321538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9db558d7-3e09-4ff8-8fd7-b0cfc223de73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:43.372233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b77820d0-3d5b-46ca-aa4c-db445b674149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57a6c9d662374991a0dd148e40af864c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:43.372897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b77820d0-3d5b-46ca-aa4c-db445b674149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57a6c9d662374991a0dd148e40af864c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:43.383967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b77820d0-3d5b-46ca-aa4c-db445b674149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57a6c9d662374991a0dd148e40af864c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:43.393219 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1346/2703] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:55:42 2022] DELETE /identity/v3/users/57a6c9d662374991a0dd148e40af864c => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:55:43.398718 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292759d-d2f5-434d-b095-284e61453a6b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:43.399301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1358/2704] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:43.399430 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292759d-d2f5-434d-b095-284e61453a6b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:43.399705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292759d-d2f5-434d-b095-284e61453a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:43.409719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db558d7-3e09-4ff8-8fd7-b0cfc223de73 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:43.411391 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9db558d7-3e09-4ff8-8fd7-b0cfc223de73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:43.424044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b292759d-d2f5-434d-b095-284e61453a6b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:43.435026 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b292759d-d2f5-434d-b095-284e61453a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:43.437432 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:43.479203 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db558d7-3e09-4ff8-8fd7-b0cfc223de73 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:43.499989 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1347/2705] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:55:43.510643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292759d-d2f5-434d-b095-284e61453a6b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:43.533007 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16763c43-87f6-4a2e-9b40-3f87f2638db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:43.533637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763c43-87f6-4a2e-9b40-3f87f2638db3 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:43.533994 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763c43-87f6-4a2e-9b40-3f87f2638db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:43.534244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16763c43-87f6-4a2e-9b40-3f87f2638db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:43.554433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16763c43-87f6-4a2e-9b40-3f87f2638db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:43.555948 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16763c43-87f6-4a2e-9b40-3f87f2638db3 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:43.567744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1359/2706] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:55:43.585099 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292759d-d2f5-434d-b095-284e61453a6b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:43.589316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7558755-c1bc-45f3-b65c-526c9a015fd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:43.590007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7558755-c1bc-45f3-b65c-526c9a015fd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:43.590329 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7558755-c1bc-45f3-b65c-526c9a015fd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:43.590608 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7558755-c1bc-45f3-b65c-526c9a015fd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:43.600618 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7558755-c1bc-45f3-b65c-526c9a015fd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:43.602105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7558755-c1bc-45f3-b65c-526c9a015fd3 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:43.614287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1348/2707] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:55:44.308500 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16763c43-87f6-4a2e-9b40-3f87f2638db3 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:44.311457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-16763c43-87f6-4a2e-9b40-3f87f2638db3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:44.312129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7558755-c1bc-45f3-b65c-526c9a015fd3 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:44.315229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f7558755-c1bc-45f3-b65c-526c9a015fd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:44.345521 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7558755-c1bc-45f3-b65c-526c9a015fd3 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:44.351355 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16763c43-87f6-4a2e-9b40-3f87f2638db3 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:44.357664 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:44.358509 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:44.358962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:44.359930 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] PATH_INFO: `/v3/projects/1d249b7417674eed8d0357bb300c07ec` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:44.361123 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d249b7417674eed8d0357bb300c07ec)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:44.363651 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:44.373235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c637c-b798-4a43-93a4-33908898dc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:44.373235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c637c-b798-4a43-93a4-33908898dc23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:44.373235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c637c-b798-4a43-93a4-33908898dc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:44.373235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c637c-b798-4a43-93a4-33908898dc23 admin admin] PATH_INFO: `/v3/projects/628b91e13fdd4830ab5d768b7dd794a5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:44.376048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c637c-b798-4a43-93a4-33908898dc23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=628b91e13fdd4830ab5d768b7dd794a5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:44.378805 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c637c-b798-4a43-93a4-33908898dc23 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:44.402786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:44.621103 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d249b7417674eed8d0357bb300c07ec', 'request_id': 'req-dd314531-49f6-4b86-9eff-2d4021520c41'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:44.677789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca6c637c-b798-4a43-93a4-33908898dc23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '628b91e13fdd4830ab5d768b7dd794a5', 'request_id': 'req-ca6c637c-b798-4a43-93a4-33908898dc23'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:44.704506 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d249b7417674eed8d0357bb300c07ec', 'request_id': 'req-dd314531-49f6-4b86-9eff-2d4021520c41'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:44.712743 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d249b7417674eed8d0357bb300c07ec', 'request_id': 'req-dd314531-49f6-4b86-9eff-2d4021520c41'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:44.727750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] The token cache is being invalidate because project 1d249b7417674eed8d0357bb300c07ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:55:44.728092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:44.728867 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:44.730713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1360/2708] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:55:44 2022] DELETE /identity/v3/projects/1d249b7417674eed8d0357bb300c07ec => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:44.736233 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:44.739187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:44.790052 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca6c637c-b798-4a43-93a4-33908898dc23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '628b91e13fdd4830ab5d768b7dd794a5', 'request_id': 'req-ca6c637c-b798-4a43-93a4-33908898dc23'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:44.800664 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca6c637c-b798-4a43-93a4-33908898dc23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '628b91e13fdd4830ab5d768b7dd794a5', 'request_id': 'req-ca6c637c-b798-4a43-93a4-33908898dc23'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:44.815198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca6c637c-b798-4a43-93a4-33908898dc23 admin admin] The token cache is being invalidate because project 628b91e13fdd4830ab5d768b7dd794a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:55:44.815588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca6c637c-b798-4a43-93a4-33908898dc23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:44.816437 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca6c637c-b798-4a43-93a4-33908898dc23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:44.818373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1349/2709] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:55:44 2022] DELETE /identity/v3/projects/628b91e13fdd4830ab5d768b7dd794a5 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:44.828172 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ad43a-c8a8-434d-af5d-990a5d484702 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:44.828426 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ad43a-c8a8-434d-af5d-990a5d484702 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:44.828689 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ad43a-c8a8-434d-af5d-990a5d484702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:44.841171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd314531-49f6-4b86-9eff-2d4021520c41 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:44.848499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:44.876760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d02ad43a-c8a8-434d-af5d-990a5d484702 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:44.880504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d02ad43a-c8a8-434d-af5d-990a5d484702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:44.943115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3379c2-7d77-4875-9401-9ffd0748a605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:44.944109 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3379c2-7d77-4875-9401-9ffd0748a605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:44.944417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3379c2-7d77-4875-9401-9ffd0748a605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:44.944693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3379c2-7d77-4875-9401-9ffd0748a605 admin admin] PATH_INFO: `/v3/users/a233f3d7c447404fba6371dec4b4da50` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:44.945984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3379c2-7d77-4875-9401-9ffd0748a605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a233f3d7c447404fba6371dec4b4da50)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:44.948407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3379c2-7d77-4875-9401-9ffd0748a605 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:44.972653 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1350/2710] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:44.981616 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237998b-1ced-45ee-bf63-df2628fb8b0b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:44.981883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237998b-1ced-45ee-bf63-df2628fb8b0b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:44.982191 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237998b-1ced-45ee-bf63-df2628fb8b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:45.006298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2237998b-1ced-45ee-bf63-df2628fb8b0b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:45.007639 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2237998b-1ced-45ee-bf63-df2628fb8b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:45.016083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1351/2711] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:45.021122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237998b-1ced-45ee-bf63-df2628fb8b0b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:45.031538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237998b-1ced-45ee-bf63-df2628fb8b0b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:45.037257 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f144b-acf0-478c-a793-f26ef4aee16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:45.038060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f144b-acf0-478c-a793-f26ef4aee16d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:45.038375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f144b-acf0-478c-a793-f26ef4aee16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:45.040543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f144b-acf0-478c-a793-f26ef4aee16d admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:45.040543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f144b-acf0-478c-a793-f26ef4aee16d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:55:45.040543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f144b-acf0-478c-a793-f26ef4aee16d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:45.041484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f144b-acf0-478c-a793-f26ef4aee16d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:45.057952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1352/2712] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:55:45.064662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f144b-acf0-478c-a793-f26ef4aee16d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:45.075187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f144b-acf0-478c-a793-f26ef4aee16d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:45.080520 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54320ac4-4663-4412-9d03-37df2318b24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:45.081581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54320ac4-4663-4412-9d03-37df2318b24e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:45.081857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54320ac4-4663-4412-9d03-37df2318b24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:45.082129 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54320ac4-4663-4412-9d03-37df2318b24e admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:45.082913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54320ac4-4663-4412-9d03-37df2318b24e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:45.084899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54320ac4-4663-4412-9d03-37df2318b24e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:45.150110 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6a3379c2-7d77-4875-9401-9ffd0748a605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a233f3d7c447404fba6371dec4b4da50'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:45.153255 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6a3379c2-7d77-4875-9401-9ffd0748a605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a233f3d7c447404fba6371dec4b4da50'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:45.211366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1353/2713] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:55:45 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:55:45.212579 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54320ac4-4663-4412-9d03-37df2318b24e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:45.214373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-54320ac4-4663-4412-9d03-37df2318b24e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:45.229172 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6a3379c2-7d77-4875-9401-9ffd0748a605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a233f3d7c447404fba6371dec4b4da50'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:45.243095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6a3379c2-7d77-4875-9401-9ffd0748a605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a233f3d7c447404fba6371dec4b4da50'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:45.265229 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6a3379c2-7d77-4875-9401-9ffd0748a605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a233f3d7c447404fba6371dec4b4da50'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:45.276903 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1361/2714] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:55:44 2022] DELETE /identity/v3/users/a233f3d7c447404fba6371dec4b4da50 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:55:45.294429 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:45.300719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54320ac4-4663-4412-9d03-37df2318b24e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:45.379363 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc72b0c-6c75-4bab-b186-f565d04bd0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:45.380411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc72b0c-6c75-4bab-b186-f565d04bd0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:45.380411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc72b0c-6c75-4bab-b186-f565d04bd0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:45.380411 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc72b0c-6c75-4bab-b186-f565d04bd0b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:45.381082 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc72b0c-6c75-4bab-b186-f565d04bd0b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:45.383254 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc72b0c-6c75-4bab-b186-f565d04bd0b0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:45.570135 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1354/2715] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:55:45 2022] POST /identity/v3/users => generated 289 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:55:45.576379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3379c2-7d77-4875-9401-9ffd0748a605 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:45.589546 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3379c2-7d77-4875-9401-9ffd0748a605 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:45.596932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af3025-8d87-4ebc-885a-2d5d333dcff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:45.597725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af3025-8d87-4ebc-885a-2d5d333dcff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:45.598195 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af3025-8d87-4ebc-885a-2d5d333dcff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:45.598661 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af3025-8d87-4ebc-885a-2d5d333dcff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:45.610351 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af3025-8d87-4ebc-885a-2d5d333dcff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:45.613154 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af3025-8d87-4ebc-885a-2d5d333dcff9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:45.631768 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1362/2716] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:45.635173 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc72b0c-6c75-4bab-b186-f565d04bd0b0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:45.655280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc72b0c-6c75-4bab-b186-f565d04bd0b0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:45.661315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c81ce-b130-44ac-8528-6bc5ed6c53e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:45.668158 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c81ce-b130-44ac-8528-6bc5ed6c53e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:45.668158 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c81ce-b130-44ac-8528-6bc5ed6c53e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:45.668158 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c81ce-b130-44ac-8528-6bc5ed6c53e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:45.668158 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c81ce-b130-44ac-8528-6bc5ed6c53e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:45.668158 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c81ce-b130-44ac-8528-6bc5ed6c53e0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:45.677973 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1355/2717] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:45.682460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af3025-8d87-4ebc-885a-2d5d333dcff9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:45.693519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af3025-8d87-4ebc-885a-2d5d333dcff9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:45.698602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64347b7f-cd88-4596-9c09-6a4bdb9dcb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:45.699281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64347b7f-cd88-4596-9c09-6a4bdb9dcb6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:45.699560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64347b7f-cd88-4596-9c09-6a4bdb9dcb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:45.699835 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64347b7f-cd88-4596-9c09-6a4bdb9dcb6c admin admin] PATH_INFO: `/v3/projects/6d5504106ccb4549b09f29d7104d9b0a/users/ed50f5d8261d44b487390de092ef9136/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:45.722055 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64347b7f-cd88-4596-9c09-6a4bdb9dcb6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d5504106ccb4549b09f29d7104d9b0a, user_id=ed50f5d8261d44b487390de092ef9136, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:45.724814 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64347b7f-cd88-4596-9c09-6a4bdb9dcb6c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:45.890629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1363/2718] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:45 2022] PUT /identity/v3/projects/6d5504106ccb4549b09f29d7104d9b0a/users/ed50f5d8261d44b487390de092ef9136/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:45.897086 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b22fc-7af0-4635-873a-190f616e7b95 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:45.897599 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b22fc-7af0-4635-873a-190f616e7b95 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:45.898178 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b22fc-7af0-4635-873a-190f616e7b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:45.923922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-158b22fc-7af0-4635-873a-190f616e7b95 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:45.925239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-158b22fc-7af0-4635-873a-190f616e7b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:46.035025 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1356/2719] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:46.047382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64347b7f-cd88-4596-9c09-6a4bdb9dcb6c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:46.050212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-64347b7f-cd88-4596-9c09-6a4bdb9dcb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:46.132461 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64347b7f-cd88-4596-9c09-6a4bdb9dcb6c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:46.136624 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0b714-40ac-428e-9576-4f59d63c02cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:46.137246 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0b714-40ac-428e-9576-4f59d63c02cc service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:46.137524 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0b714-40ac-428e-9576-4f59d63c02cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:46.137964 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0b714-40ac-428e-9576-4f59d63c02cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:46.148642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:46.150809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b22fc-7af0-4635-873a-190f616e7b95 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:46.152111 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0b714-40ac-428e-9576-4f59d63c02cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:46.153201 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-158b22fc-7af0-4635-873a-190f616e7b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:46.153963 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0b714-40ac-428e-9576-4f59d63c02cc service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:46.167335 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1364/2720] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:55:46.174281 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b22fc-7af0-4635-873a-190f616e7b95 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:46.185092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46794b-e72d-4dcd-adce-34195278df78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:46.186243 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46794b-e72d-4dcd-adce-34195278df78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:46.187496 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46794b-e72d-4dcd-adce-34195278df78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:46.187993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46794b-e72d-4dcd-adce-34195278df78 admin admin] PATH_INFO: `/v3/projects/4577fdb978d94d62b4fbc632a18b9f71` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:46.190919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46794b-e72d-4dcd-adce-34195278df78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4577fdb978d94d62b4fbc632a18b9f71)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:46.195751 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46794b-e72d-4dcd-adce-34195278df78 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:46.369822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9a46794b-e72d-4dcd-adce-34195278df78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4577fdb978d94d62b4fbc632a18b9f71', 'request_id': 'req-9a46794b-e72d-4dcd-adce-34195278df78'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:46.488865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9a46794b-e72d-4dcd-adce-34195278df78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4577fdb978d94d62b4fbc632a18b9f71', 'request_id': 'req-9a46794b-e72d-4dcd-adce-34195278df78'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:46.500556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9a46794b-e72d-4dcd-adce-34195278df78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4577fdb978d94d62b4fbc632a18b9f71', 'request_id': 'req-9a46794b-e72d-4dcd-adce-34195278df78'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:46.524235 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9a46794b-e72d-4dcd-adce-34195278df78 admin admin] The token cache is being invalidate because project 4577fdb978d94d62b4fbc632a18b9f71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:55:46.524692 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9a46794b-e72d-4dcd-adce-34195278df78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:46.525594 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9a46794b-e72d-4dcd-adce-34195278df78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:55:46.527621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1357/2721] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:55:46 2022] DELETE /identity/v3/projects/4577fdb978d94d62b4fbc632a18b9f71 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:46.533722 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddd7f6-7325-4d42-a25d-7153494108b6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:46.534618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddd7f6-7325-4d42-a25d-7153494108b6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:46.535279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddd7f6-7325-4d42-a25d-7153494108b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:46.566976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b0ddd7f6-7325-4d42-a25d-7153494108b6 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:46.568243 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b0ddd7f6-7325-4d42-a25d-7153494108b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:46.652334 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1365/2722] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:55:46.667013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e22e93-b7f8-4e37-9a39-3054cd4f5627 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:46.667530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e22e93-b7f8-4e37-9a39-3054cd4f5627 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:46.668003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e22e93-b7f8-4e37-9a39-3054cd4f5627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:46.686842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:46.694544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c7e22e93-b7f8-4e37-9a39-3054cd4f5627 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:46.696900 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c7e22e93-b7f8-4e37-9a39-3054cd4f5627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:46.711265 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1358/2723] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:46.712149 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddd7f6-7325-4d42-a25d-7153494108b6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:46.735114 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddd7f6-7325-4d42-a25d-7153494108b6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:46.745445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44170b6-cc7f-4160-88a0-5739a3aef5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:46.746708 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44170b6-cc7f-4160-88a0-5739a3aef5ac admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:46.747315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44170b6-cc7f-4160-88a0-5739a3aef5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:46.747872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44170b6-cc7f-4160-88a0-5739a3aef5ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:46.749400 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44170b6-cc7f-4160-88a0-5739a3aef5ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:55:46.750324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44170b6-cc7f-4160-88a0-5739a3aef5ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:46.753894 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44170b6-cc7f-4160-88a0-5739a3aef5ac admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:46.781466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1366/2724] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:55:46.787632 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e22e93-b7f8-4e37-9a39-3054cd4f5627 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:46.828754 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e22e93-b7f8-4e37-9a39-3054cd4f5627 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:46.837115 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e42d27-2855-4622-8621-505668596953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:46.838177 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42d27-2855-4622-8621-505668596953 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:46.838688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42d27-2855-4622-8621-505668596953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:46.839171 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42d27-2855-4622-8621-505668596953 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:46.840517 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e42d27-2855-4622-8621-505668596953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:46.844474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e42d27-2855-4622-8621-505668596953 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:47.022769 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1359/2725] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:55:46 2022] POST /identity/v3/projects => generated 365 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:55:47.027325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44170b6-cc7f-4160-88a0-5739a3aef5ac admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:47.058056 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44170b6-cc7f-4160-88a0-5739a3aef5ac admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:47.150645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a4b50-5a3c-48a6-8a0f-a7915a3caf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:47.151841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a4b50-5a3c-48a6-8a0f-a7915a3caf8f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:47.152648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a4b50-5a3c-48a6-8a0f-a7915a3caf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:47.153239 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a4b50-5a3c-48a6-8a0f-a7915a3caf8f admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:47.155310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a4b50-5a3c-48a6-8a0f-a7915a3caf8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:47.160339 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a4b50-5a3c-48a6-8a0f-a7915a3caf8f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:47.300654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1367/2726] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:55:47 2022] POST /identity/v3/users => generated 283 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:55:47.302331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e42d27-2855-4622-8621-505668596953 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:47.314949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e42d27-2855-4622-8621-505668596953 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:47.321301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712da7b-a9d8-4303-9f87-7f325fa7c463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:47.322385 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712da7b-a9d8-4303-9f87-7f325fa7c463 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:47.323028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712da7b-a9d8-4303-9f87-7f325fa7c463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:47.323565 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712da7b-a9d8-4303-9f87-7f325fa7c463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:47.324486 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712da7b-a9d8-4303-9f87-7f325fa7c463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:47.326498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712da7b-a9d8-4303-9f87-7f325fa7c463 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:47.338815 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1360/2727] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:47.354808 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a4b50-5a3c-48a6-8a0f-a7915a3caf8f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:47.373411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a4b50-5a3c-48a6-8a0f-a7915a3caf8f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:47.381050 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bbfe28-7c9f-4c63-b11f-3cddefbbde3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:47.381882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbfe28-7c9f-4c63-b11f-3cddefbbde3e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:47.382293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbfe28-7c9f-4c63-b11f-3cddefbbde3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:47.382698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbfe28-7c9f-4c63-b11f-3cddefbbde3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:47.383452 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bbfe28-7c9f-4c63-b11f-3cddefbbde3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:47.385238 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bbfe28-7c9f-4c63-b11f-3cddefbbde3e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:47.399499 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1368/2728] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:47.406109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712da7b-a9d8-4303-9f87-7f325fa7c463 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:47.426315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712da7b-a9d8-4303-9f87-7f325fa7c463 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:47.433359 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2589fe-24a6-42fa-940b-dfb48867c7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:47.434095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2589fe-24a6-42fa-940b-dfb48867c7ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:47.434502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2589fe-24a6-42fa-940b-dfb48867c7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:47.435592 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2589fe-24a6-42fa-940b-dfb48867c7ec admin admin] PATH_INFO: `/v3/projects/0227dfbcdd18414ba064b0af6e924820/users/d70c69b204b24428afbd2c1c39f127e3/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:47.466822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2589fe-24a6-42fa-940b-dfb48867c7ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=0227dfbcdd18414ba064b0af6e924820, user_id=d70c69b204b24428afbd2c1c39f127e3, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:47.467040 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2589fe-24a6-42fa-940b-dfb48867c7ec admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:47.592515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1361/2729] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:47 2022] PUT /identity/v3/projects/0227dfbcdd18414ba064b0af6e924820/users/d70c69b204b24428afbd2c1c39f127e3/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:47.605675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2384e14-60c1-41ee-b98e-0d7b23b7c51d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:47.606922 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2384e14-60c1-41ee-b98e-0d7b23b7c51d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:47.607295 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2384e14-60c1-41ee-b98e-0d7b23b7c51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:47.648640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f2384e14-60c1-41ee-b98e-0d7b23b7c51d None None] MFA Rules not processed for user `d70c69b204b24428afbd2c1c39f127e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:47.650317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f2384e14-60c1-41ee-b98e-0d7b23b7c51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:47.781855 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:47.812032 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1369/2730] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:47.814093 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2589fe-24a6-42fa-940b-dfb48867c7ec admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:47.910454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2589fe-24a6-42fa-940b-dfb48867c7ec admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:47.916854 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abb62e-e10b-4521-9765-7205946210a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:47.918605 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abb62e-e10b-4521-9765-7205946210a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:47.918605 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abb62e-e10b-4521-9765-7205946210a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:47.918968 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abb62e-e10b-4521-9765-7205946210a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:47.925434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abb62e-e10b-4521-9765-7205946210a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:47.925434 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abb62e-e10b-4521-9765-7205946210a5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:48.063056 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1362/2731] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:55:47 2022] POST /identity/v3/projects => generated 365 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:55:48.066398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2384e14-60c1-41ee-b98e-0d7b23b7c51d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:48.086579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2384e14-60c1-41ee-b98e-0d7b23b7c51d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:48.207597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67e1c7-7e9e-4b87-b2fa-e2fa3587aeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:48.208268 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67e1c7-7e9e-4b87-b2fa-e2fa3587aeaa admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:48.208543 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67e1c7-7e9e-4b87-b2fa-e2fa3587aeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:48.208812 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67e1c7-7e9e-4b87-b2fa-e2fa3587aeaa admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:48.209468 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67e1c7-7e9e-4b87-b2fa-e2fa3587aeaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:48.211649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67e1c7-7e9e-4b87-b2fa-e2fa3587aeaa admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:48.477129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1370/2732] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:55:48 2022] POST /identity/v3/users => generated 289 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:55:48.478558 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abb62e-e10b-4521-9765-7205946210a5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:48.490390 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abb62e-e10b-4521-9765-7205946210a5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:48.496122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad993a-cefa-4a91-84d5-99a929ab78fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:48.497032 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad993a-cefa-4a91-84d5-99a929ab78fc admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:48.497473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad993a-cefa-4a91-84d5-99a929ab78fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:48.497929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad993a-cefa-4a91-84d5-99a929ab78fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:48.498619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad993a-cefa-4a91-84d5-99a929ab78fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:48.500226 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad993a-cefa-4a91-84d5-99a929ab78fc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:48.514182 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1363/2733] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:55:48.519052 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67e1c7-7e9e-4b87-b2fa-e2fa3587aeaa admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:48.530291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67e1c7-7e9e-4b87-b2fa-e2fa3587aeaa admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:48.536542 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e1691-8d21-4c7d-919a-65feea7b6b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:48.537354 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e1691-8d21-4c7d-919a-65feea7b6b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:48.537697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e1691-8d21-4c7d-919a-65feea7b6b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:48.538190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e1691-8d21-4c7d-919a-65feea7b6b96 admin admin] PATH_INFO: `/v3/projects/858064a293d448528f954f0f4ccdce48/users/54f337a836064e60af0404a282eb0115/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:48.558870 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e1691-8d21-4c7d-919a-65feea7b6b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=858064a293d448528f954f0f4ccdce48, user_id=54f337a836064e60af0404a282eb0115, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:48.562791 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e1691-8d21-4c7d-919a-65feea7b6b96 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:48.668796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1371/2734] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:48 2022] PUT /identity/v3/projects/858064a293d448528f954f0f4ccdce48/users/54f337a836064e60af0404a282eb0115/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:55:48.671009 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad993a-cefa-4a91-84d5-99a929ab78fc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:48.695891 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad993a-cefa-4a91-84d5-99a929ab78fc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:48.794943 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311f193-e234-4b8e-9bba-44656d77e8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:48.795994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311f193-e234-4b8e-9bba-44656d77e8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:48.796460 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311f193-e234-4b8e-9bba-44656d77e8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:48.796913 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311f193-e234-4b8e-9bba-44656d77e8c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:48.797766 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311f193-e234-4b8e-9bba-44656d77e8c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:48.800563 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311f193-e234-4b8e-9bba-44656d77e8c7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:48.818974 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1364/2735] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:48.823743 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e1691-8d21-4c7d-919a-65feea7b6b96 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:48.843183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e1691-8d21-4c7d-919a-65feea7b6b96 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:48.851178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9203e-b427-4c97-b32b-daabd060f175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:48.852968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9203e-b427-4c97-b32b-daabd060f175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:48.853456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9203e-b427-4c97-b32b-daabd060f175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:48.854034 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9203e-b427-4c97-b32b-daabd060f175 admin admin] PATH_INFO: `/v3/projects/858064a293d448528f954f0f4ccdce48/users/54f337a836064e60af0404a282eb0115/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:48.863096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9203e-b427-4c97-b32b-daabd060f175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=858064a293d448528f954f0f4ccdce48, user_id=54f337a836064e60af0404a282eb0115, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:48.868494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9203e-b427-4c97-b32b-daabd060f175 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:48.894033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1372/2736] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:48 2022] PUT /identity/v3/projects/858064a293d448528f954f0f4ccdce48/users/54f337a836064e60af0404a282eb0115/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:55:48.904734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6cfd9-34c3-4231-bf0f-a1201178b1f4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:48.905311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6cfd9-34c3-4231-bf0f-a1201178b1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:48.905752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6cfd9-34c3-4231-bf0f-a1201178b1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:48.935345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:48.946426 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-97f6cfd9-34c3-4231-bf0f-a1201178b1f4 None None] MFA Rules not processed for user `54f337a836064e60af0404a282eb0115`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:48.948523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-97f6cfd9-34c3-4231-bf0f-a1201178b1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:49.085880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1365/2737] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:55:49.094688 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9203e-b427-4c97-b32b-daabd060f175 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:49.097985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-87f9203e-b427-4c97-b32b-daabd060f175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:49.185680 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9203e-b427-4c97-b32b-daabd060f175 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:49.190959 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef647c54-b242-4f9d-929d-7fb06b8ee54f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:49.194387 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef647c54-b242-4f9d-929d-7fb06b8ee54f service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:49.194785 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef647c54-b242-4f9d-929d-7fb06b8ee54f service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:49.195174 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef647c54-b242-4f9d-929d-7fb06b8ee54f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:49.206271 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef647c54-b242-4f9d-929d-7fb06b8ee54f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:49.208591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef647c54-b242-4f9d-929d-7fb06b8ee54f service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:49.220831 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1373/2738] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:55:49.578971 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6cfd9-34c3-4231-bf0f-a1201178b1f4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:49.589970 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6cfd9-34c3-4231-bf0f-a1201178b1f4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:49.609823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9eebdc-f806-45b3-af3e-a3d99c8bc1a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:49.618050 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eebdc-f806-45b3-af3e-a3d99c8bc1a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:49.618770 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eebdc-f806-45b3-af3e-a3d99c8bc1a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:49.619954 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eebdc-f806-45b3-af3e-a3d99c8bc1a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:49.751312 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9eebdc-f806-45b3-af3e-a3d99c8bc1a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:49.753477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9eebdc-f806-45b3-af3e-a3d99c8bc1a4 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:49.763896 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1366/2739] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:55:50.649048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef647c54-b242-4f9d-929d-7fb06b8ee54f service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:50.650849 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ef647c54-b242-4f9d-929d-7fb06b8ee54f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:50.738358 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:50.750662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef647c54-b242-4f9d-929d-7fb06b8ee54f service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:50.756854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6d965-19fa-4fd7-a02f-b77053ac1960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:50.757955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6d965-19fa-4fd7-a02f-b77053ac1960 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:50.758553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6d965-19fa-4fd7-a02f-b77053ac1960 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:50.759070 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6d965-19fa-4fd7-a02f-b77053ac1960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:50.761450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5cf6d965-19fa-4fd7-a02f-b77053ac1960 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:50.875217 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6d965-19fa-4fd7-a02f-b77053ac1960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:50.877942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6d965-19fa-4fd7-a02f-b77053ac1960 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:50.931512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1374/2740] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:55:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:55:53.899734 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641a543-8aff-4b9e-bd83-f4280e9f90ee None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:53.900607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641a543-8aff-4b9e-bd83-f4280e9f90ee None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:53.900902 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641a543-8aff-4b9e-bd83-f4280e9f90ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:53.956120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1641a543-8aff-4b9e-bd83-f4280e9f90ee None None] MFA Rules not processed for user `c1c1de0489104713a90c59b89a284321`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:53.968686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1641a543-8aff-4b9e-bd83-f4280e9f90ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:54.184884 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1367/2741] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:55:56.491976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b38d9-ade3-455e-860c-a163e296af9e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:56.491976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b38d9-ade3-455e-860c-a163e296af9e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:56.491976 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b38d9-ade3-455e-860c-a163e296af9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:56.550216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-045b38d9-ade3-455e-860c-a163e296af9e None None] MFA Rules not processed for user `ed50f5d8261d44b487390de092ef9136`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:55:56.551680 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-045b38d9-ade3-455e-860c-a163e296af9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:56.688889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1375/2742] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:55:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:55:56.695204 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641a543-8aff-4b9e-bd83-f4280e9f90ee None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:56.697455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1641a543-8aff-4b9e-bd83-f4280e9f90ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:55:56.759635 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:56.800149 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641a543-8aff-4b9e-bd83-f4280e9f90ee None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:56.803291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d333e95-f1e5-41f6-89b5-22a0d41b82ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:56.804044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d333e95-f1e5-41f6-89b5-22a0d41b82ae admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:56.804330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d333e95-f1e5-41f6-89b5-22a0d41b82ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:56.804630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d333e95-f1e5-41f6-89b5-22a0d41b82ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:56.805321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d333e95-f1e5-41f6-89b5-22a0d41b82ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:56.808666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d333e95-f1e5-41f6-89b5-22a0d41b82ae admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:56.972961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1368/2743] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:55:56 2022] POST /identity/v3/projects => generated 377 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:55:56.979913 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b38d9-ade3-455e-860c-a163e296af9e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:57.007282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b38d9-ade3-455e-860c-a163e296af9e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:57.169853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc119f2-5886-406b-8af3-81bc7e9e6cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:57.170700 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc119f2-5886-406b-8af3-81bc7e9e6cbd admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:57.171159 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc119f2-5886-406b-8af3-81bc7e9e6cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:57.171615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc119f2-5886-406b-8af3-81bc7e9e6cbd admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:57.172725 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc119f2-5886-406b-8af3-81bc7e9e6cbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:57.176897 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc119f2-5886-406b-8af3-81bc7e9e6cbd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:57.293698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1376/2744] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:55:56 2022] POST /identity/v3/users => generated 295 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:55:57.299934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d333e95-f1e5-41f6-89b5-22a0d41b82ae admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:57.315755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d333e95-f1e5-41f6-89b5-22a0d41b82ae admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:57.323388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea8fe0-2773-444b-a550-686182cf9980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:57.324524 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea8fe0-2773-444b-a550-686182cf9980 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:57.325273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea8fe0-2773-444b-a550-686182cf9980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:57.325933 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea8fe0-2773-444b-a550-686182cf9980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:57.327028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea8fe0-2773-444b-a550-686182cf9980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:57.329909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea8fe0-2773-444b-a550-686182cf9980 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:57.344947 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1369/2745] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:57.351433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc119f2-5886-406b-8af3-81bc7e9e6cbd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:57.371487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc119f2-5886-406b-8af3-81bc7e9e6cbd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:57.380182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f50918-e2c5-4816-94e9-cabf2b6021a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:57.383404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f50918-e2c5-4816-94e9-cabf2b6021a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:57.383404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f50918-e2c5-4816-94e9-cabf2b6021a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:57.383404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f50918-e2c5-4816-94e9-cabf2b6021a3 admin admin] PATH_INFO: `/v3/projects/462948c022644502b584955878a2adbc/users/dc14bedd0ece41ab953ce98db4ab5abe/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:57.412118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f50918-e2c5-4816-94e9-cabf2b6021a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=462948c022644502b584955878a2adbc, user_id=dc14bedd0ece41ab953ce98db4ab5abe, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:57.417972 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f50918-e2c5-4816-94e9-cabf2b6021a3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:57.425143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:55:57.548268 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1377/2746] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:57 2022] PUT /identity/v3/projects/462948c022644502b584955878a2adbc/users/dc14bedd0ece41ab953ce98db4ab5abe/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:55:57.564560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea8fe0-2773-444b-a550-686182cf9980 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:57.582567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea8fe0-2773-444b-a550-686182cf9980 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:57.690945 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff531e44-ada4-4c9b-aa4d-d1c7af922b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:57.691969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff531e44-ada4-4c9b-aa4d-d1c7af922b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:57.692456 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff531e44-ada4-4c9b-aa4d-d1c7af922b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:57.692955 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff531e44-ada4-4c9b-aa4d-d1c7af922b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:57.694123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff531e44-ada4-4c9b-aa4d-d1c7af922b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:57.696922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff531e44-ada4-4c9b-aa4d-d1c7af922b4e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:57.715130 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1370/2747] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:55:57.720080 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f50918-e2c5-4816-94e9-cabf2b6021a3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:55:57.733603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f50918-e2c5-4816-94e9-cabf2b6021a3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:55:57.741606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a821ef-3ab4-404a-a88f-63d78cf89aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:55:57.742446 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a821ef-3ab4-404a-a88f-63d78cf89aed admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:55:57.742796 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a821ef-3ab4-404a-a88f-63d78cf89aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:55:57.743135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a821ef-3ab4-404a-a88f-63d78cf89aed admin admin] PATH_INFO: `/v3/projects/462948c022644502b584955878a2adbc/users/dc14bedd0ece41ab953ce98db4ab5abe/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:55:57.745335 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a821ef-3ab4-404a-a88f-63d78cf89aed admin admin] RBAC: Authorizing `identity:create_grant(project_id=462948c022644502b584955878a2adbc, user_id=dc14bedd0ece41ab953ce98db4ab5abe, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:55:57.749176 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a821ef-3ab4-404a-a88f-63d78cf89aed admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:55:57.767158 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1378/2748] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:55:57 2022] PUT /identity/v3/projects/462948c022644502b584955878a2adbc/users/dc14bedd0ece41ab953ce98db4ab5abe/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:02.666927 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff531e44-ada4-4c9b-aa4d-d1c7af922b4e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:02.675331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ff531e44-ada4-4c9b-aa4d-d1c7af922b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:02.804252 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff531e44-ada4-4c9b-aa4d-d1c7af922b4e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:02.879010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d0f16-6399-42a0-bdc6-acecf0bcd8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:02.879669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d0f16-6399-42a0-bdc6-acecf0bcd8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:02.879949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d0f16-6399-42a0-bdc6-acecf0bcd8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:02.880223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d0f16-6399-42a0-bdc6-acecf0bcd8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:02.882243 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ff6d0f16-6399-42a0-bdc6-acecf0bcd8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:02.917506 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:03.032442 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d0f16-6399-42a0-bdc6-acecf0bcd8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:03.036152 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d0f16-6399-42a0-bdc6-acecf0bcd8e9 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:03.053445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1371/2749] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:56:05.153377 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a821ef-3ab4-404a-a88f-63d78cf89aed admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:05.178457 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a821ef-3ab4-404a-a88f-63d78cf89aed admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:05.195254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:05.198265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:05.199347 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:05.200111 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] PATH_INFO: `/v3/users/d70c69b204b24428afbd2c1c39f127e3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:05.203236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d70c69b204b24428afbd2c1c39f127e3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:05.211570 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:05.624917 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd70c69b204b24428afbd2c1c39f127e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:05.627301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd70c69b204b24428afbd2c1c39f127e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:05.717122 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd70c69b204b24428afbd2c1c39f127e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:05.730293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd70c69b204b24428afbd2c1c39f127e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:05.768055 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd70c69b204b24428afbd2c1c39f127e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:05.782639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1379/2750] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:56:05 2022] DELETE /identity/v3/users/d70c69b204b24428afbd2c1c39f127e3 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:56:05.788695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bed1e-d525-457c-bde7-1e48e84b2070 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:05.788695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bed1e-d525-457c-bde7-1e48e84b2070 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:05.788695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bed1e-d525-457c-bde7-1e48e84b2070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:05.826842 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d90bed1e-d525-457c-bde7-1e48e84b2070 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:05.829969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d90bed1e-d525-457c-bde7-1e48e84b2070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:05.943601 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1372/2751] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:05.986966 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:06.002450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:06.094168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87cdf1-5bc0-4c77-919f-42c690c32d50 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:06.105164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fddb1-ebb2-4dd3-b4d2-b6aba68a2a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:06.106928 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fddb1-ebb2-4dd3-b4d2-b6aba68a2a94 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:06.107225 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fddb1-ebb2-4dd3-b4d2-b6aba68a2a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:06.107503 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fddb1-ebb2-4dd3-b4d2-b6aba68a2a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:06.128113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fddb1-ebb2-4dd3-b4d2-b6aba68a2a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:06.129973 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fddb1-ebb2-4dd3-b4d2-b6aba68a2a94 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:06.146163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1380/2752] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:56:07.548057 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bed1e-d525-457c-bde7-1e48e84b2070 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:07.549682 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d90bed1e-d525-457c-bde7-1e48e84b2070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:07.564910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bed1e-d525-457c-bde7-1e48e84b2070 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:07.573626 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:07.574480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:07.575987 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:07.576348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab admin admin] PATH_INFO: `/v3/projects/0227dfbcdd18414ba064b0af6e924820` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:07.578996 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0227dfbcdd18414ba064b0af6e924820)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:07.582609 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:07.801296 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0227dfbcdd18414ba064b0af6e924820', 'request_id': 'req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:07.904098 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0227dfbcdd18414ba064b0af6e924820', 'request_id': 'req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:07.918180 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0227dfbcdd18414ba064b0af6e924820', 'request_id': 'req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:07.962944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab admin admin] The token cache is being invalidate because project 0227dfbcdd18414ba064b0af6e924820 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:56:07.962944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:07.962944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa0406e7-5863-4a25-90a5-8bb9d52f16ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:07.962944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1373/2753] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:56:07 2022] DELETE /identity/v3/projects/0227dfbcdd18414ba064b0af6e924820 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:07.964462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fddb1-ebb2-4dd3-b4d2-b6aba68a2a94 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:07.981106 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fddb1-ebb2-4dd3-b4d2-b6aba68a2a94 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:08.109629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:08.121177 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:08.121945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:08.122396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:08.122720 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] PATH_INFO: `/v3/users/54f337a836064e60af0404a282eb0115` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:08.123970 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54f337a836064e60af0404a282eb0115)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:08.132892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:08.451445 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54f337a836064e60af0404a282eb0115'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:08.452613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54f337a836064e60af0404a282eb0115'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:08.584072 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54f337a836064e60af0404a282eb0115'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:08.593116 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54f337a836064e60af0404a282eb0115'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:08.619643 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54f337a836064e60af0404a282eb0115'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:08.632171 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1381/2754] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:56:07 2022] DELETE /identity/v3/users/54f337a836064e60af0404a282eb0115 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:09.848393 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8a1e1-beb8-4ada-af61-31e8226e2b33 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:09.848963 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8a1e1-beb8-4ada-af61-31e8226e2b33 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:09.849162 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8a1e1-beb8-4ada-af61-31e8226e2b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:09.909013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-50a8a1e1-beb8-4ada-af61-31e8226e2b33 None None] MFA Rules not processed for user `dc14bedd0ece41ab953ce98db4ab5abe`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:09.910980 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-50a8a1e1-beb8-4ada-af61-31e8226e2b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:09.964859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:09.967059 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:10.013928 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:10.120441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c025e9-539f-4130-a281-cbd804eefb51 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:10.128627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b024e-807d-4da1-9b14-9fc298e85716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:10.129587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b024e-807d-4da1-9b14-9fc298e85716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:10.130021 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b024e-807d-4da1-9b14-9fc298e85716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:10.130482 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b024e-807d-4da1-9b14-9fc298e85716 admin admin] PATH_INFO: `/v3/projects/858064a293d448528f954f0f4ccdce48` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:10.131859 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b024e-807d-4da1-9b14-9fc298e85716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=858064a293d448528f954f0f4ccdce48)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:10.134123 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b024e-807d-4da1-9b14-9fc298e85716 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:10.208824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1374/2755] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:56:10.231455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8a1e1-beb8-4ada-af61-31e8226e2b33 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:10.234069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-50a8a1e1-beb8-4ada-af61-31e8226e2b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:10.333138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-967b024e-807d-4da1-9b14-9fc298e85716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '858064a293d448528f954f0f4ccdce48', 'request_id': 'req-967b024e-807d-4da1-9b14-9fc298e85716'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:10.378439 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8a1e1-beb8-4ada-af61-31e8226e2b33 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:10.452653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-967b024e-807d-4da1-9b14-9fc298e85716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '858064a293d448528f954f0f4ccdce48', 'request_id': 'req-967b024e-807d-4da1-9b14-9fc298e85716'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:10.461879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-967b024e-807d-4da1-9b14-9fc298e85716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '858064a293d448528f954f0f4ccdce48', 'request_id': 'req-967b024e-807d-4da1-9b14-9fc298e85716'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:10.476270 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-967b024e-807d-4da1-9b14-9fc298e85716 admin admin] The token cache is being invalidate because project 858064a293d448528f954f0f4ccdce48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:56:10.476563 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-967b024e-807d-4da1-9b14-9fc298e85716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:10.477224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-967b024e-807d-4da1-9b14-9fc298e85716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:10.485455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1382/2756] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:56:09 2022] DELETE /identity/v3/projects/858064a293d448528f954f0f4ccdce48 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:10.486425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a545c0-be57-446d-9032-60a47eee17bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:10.486425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a545c0-be57-446d-9032-60a47eee17bd service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:10.486425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a545c0-be57-446d-9032-60a47eee17bd service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:10.486425 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a545c0-be57-446d-9032-60a47eee17bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:10.487461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-01a545c0-be57-446d-9032-60a47eee17bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:10.497184 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92837aa7-f02e-4741-87c6-dd25a8fdb76b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:10.499918 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92837aa7-f02e-4741-87c6-dd25a8fdb76b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:10.502532 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92837aa7-f02e-4741-87c6-dd25a8fdb76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:10.533771 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:10.537214 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:10.547714 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-92837aa7-f02e-4741-87c6-dd25a8fdb76b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:10.551399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-92837aa7-f02e-4741-87c6-dd25a8fdb76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:10.622083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a545c0-be57-446d-9032-60a47eee17bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:10.625830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a545c0-be57-446d-9032-60a47eee17bd service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:10.650885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1375/2757] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:56:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:56:10.667127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1383/2758] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:10.678198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443383c1-006e-4dcb-ba4d-0f30b8f6bb12 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:10.679191 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443383c1-006e-4dcb-ba4d-0f30b8f6bb12 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:10.680886 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443383c1-006e-4dcb-ba4d-0f30b8f6bb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:10.706331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-443383c1-006e-4dcb-ba4d-0f30b8f6bb12 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:10.708330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-443383c1-006e-4dcb-ba4d-0f30b8f6bb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:10.722353 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1376/2759] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:10.726173 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92837aa7-f02e-4741-87c6-dd25a8fdb76b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:10.737390 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92837aa7-f02e-4741-87c6-dd25a8fdb76b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:10.744455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf8505-b4bb-43b2-afdb-359c5f8a7dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:10.745294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf8505-b4bb-43b2-afdb-359c5f8a7dcd admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:10.745671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf8505-b4bb-43b2-afdb-359c5f8a7dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:10.746045 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf8505-b4bb-43b2-afdb-359c5f8a7dcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:10.746904 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf8505-b4bb-43b2-afdb-359c5f8a7dcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:56:10.747302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf8505-b4bb-43b2-afdb-359c5f8a7dcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:10.749275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf8505-b4bb-43b2-afdb-359c5f8a7dcd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:10.770244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1384/2760] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:56:10.772556 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443383c1-006e-4dcb-ba4d-0f30b8f6bb12 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:10.788500 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443383c1-006e-4dcb-ba4d-0f30b8f6bb12 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:10.802489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b37928-4fe7-4054-8ae8-69618d1738be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:10.803118 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b37928-4fe7-4054-8ae8-69618d1738be admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:10.803427 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b37928-4fe7-4054-8ae8-69618d1738be admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:10.803704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b37928-4fe7-4054-8ae8-69618d1738be admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:10.804348 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b37928-4fe7-4054-8ae8-69618d1738be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:10.807342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b37928-4fe7-4054-8ae8-69618d1738be admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:10.948407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1377/2761] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:56:10 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:56:10.952899 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf8505-b4bb-43b2-afdb-359c5f8a7dcd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:10.984744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf8505-b4bb-43b2-afdb-359c5f8a7dcd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:11.086852 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8eca78-78e1-4b05-a022-789caad79d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:11.088193 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eca78-78e1-4b05-a022-789caad79d72 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:11.088650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eca78-78e1-4b05-a022-789caad79d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:11.089093 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8eca78-78e1-4b05-a022-789caad79d72 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:11.089904 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8eca78-78e1-4b05-a022-789caad79d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:11.094307 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8eca78-78e1-4b05-a022-789caad79d72 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:11.188196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1385/2762] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:56:10 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:56:11.192124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b37928-4fe7-4054-8ae8-69618d1738be admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:11.207825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b37928-4fe7-4054-8ae8-69618d1738be admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:11.217742 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8558e25-5401-4ebf-9127-198e03b4f2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:11.218480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8558e25-5401-4ebf-9127-198e03b4f2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:11.218958 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8558e25-5401-4ebf-9127-198e03b4f2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:11.219407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8558e25-5401-4ebf-9127-198e03b4f2f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:11.220948 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8558e25-5401-4ebf-9127-198e03b4f2f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:11.223420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8558e25-5401-4ebf-9127-198e03b4f2f6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:11.243140 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1378/2763] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:56:11.244658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8eca78-78e1-4b05-a022-789caad79d72 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:11.257606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8eca78-78e1-4b05-a022-789caad79d72 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:11.264064 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e43b4d-30f0-4dd7-b4f9-a5b2029e521e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:11.264775 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e43b4d-30f0-4dd7-b4f9-a5b2029e521e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:11.265053 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e43b4d-30f0-4dd7-b4f9-a5b2029e521e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:11.265373 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e43b4d-30f0-4dd7-b4f9-a5b2029e521e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:11.271949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e43b4d-30f0-4dd7-b4f9-a5b2029e521e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:11.277315 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e43b4d-30f0-4dd7-b4f9-a5b2029e521e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:11.310191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1386/2764] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:56:11.317287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8558e25-5401-4ebf-9127-198e03b4f2f6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:11.331491 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8558e25-5401-4ebf-9127-198e03b4f2f6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:11.339838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699260af-78a5-4f5e-a2d5-991fe865d7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:11.341097 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699260af-78a5-4f5e-a2d5-991fe865d7d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:11.341523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699260af-78a5-4f5e-a2d5-991fe865d7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:11.341960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699260af-78a5-4f5e-a2d5-991fe865d7d5 admin admin] PATH_INFO: `/v3/projects/b0526189598d47448cc04dd3ce93a863/users/c0d24c3eb92a49e39462d6b2e66ba6b3/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:11.363865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699260af-78a5-4f5e-a2d5-991fe865d7d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0526189598d47448cc04dd3ce93a863, user_id=c0d24c3eb92a49e39462d6b2e66ba6b3, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:11.367340 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699260af-78a5-4f5e-a2d5-991fe865d7d5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:11.534553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1379/2765] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:56:11 2022] PUT /identity/v3/projects/b0526189598d47448cc04dd3ce93a863/users/c0d24c3eb92a49e39462d6b2e66ba6b3/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:11.545268 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb9a9c-7664-4cec-af44-5d63737c8c5b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:11.545388 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb9a9c-7664-4cec-af44-5d63737c8c5b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:11.545674 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb9a9c-7664-4cec-af44-5d63737c8c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:11.596554 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0fcb9a9c-7664-4cec-af44-5d63737c8c5b None None] MFA Rules not processed for user `c0d24c3eb92a49e39462d6b2e66ba6b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:11.598476 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0fcb9a9c-7664-4cec-af44-5d63737c8c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:11.741983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1387/2766] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:11.748465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699260af-78a5-4f5e-a2d5-991fe865d7d5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:11.862407 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699260af-78a5-4f5e-a2d5-991fe865d7d5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:11.869263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfee633-dba3-4808-af85-a4aa304f3d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:11.869887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfee633-dba3-4808-af85-a4aa304f3d1b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:11.870208 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfee633-dba3-4808-af85-a4aa304f3d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:11.870465 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfee633-dba3-4808-af85-a4aa304f3d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:11.871218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfee633-dba3-4808-af85-a4aa304f3d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:11.873131 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfee633-dba3-4808-af85-a4aa304f3d1b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:12.015762 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1380/2767] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:56:11 2022] POST /identity/v3/projects => generated 375 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:56:12.020608 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb9a9c-7664-4cec-af44-5d63737c8c5b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:12.027138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:12.035827 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb9a9c-7664-4cec-af44-5d63737c8c5b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:12.129090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873d0fe-4caa-4caf-bdc1-e867f2e2af57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:12.130316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873d0fe-4caa-4caf-bdc1-e867f2e2af57 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:12.130612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873d0fe-4caa-4caf-bdc1-e867f2e2af57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:12.130938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873d0fe-4caa-4caf-bdc1-e867f2e2af57 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:12.131857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873d0fe-4caa-4caf-bdc1-e867f2e2af57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:12.135252 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873d0fe-4caa-4caf-bdc1-e867f2e2af57 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:12.249279 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1388/2768] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:56:12 2022] POST /identity/v3/users => generated 294 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:56:12.253198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfee633-dba3-4808-af85-a4aa304f3d1b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:12.265473 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfee633-dba3-4808-af85-a4aa304f3d1b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:12.271351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c00c58-31c7-4513-b0e6-b737145d6528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:12.272196 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c00c58-31c7-4513-b0e6-b737145d6528 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:12.272644 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c00c58-31c7-4513-b0e6-b737145d6528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:12.273241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c00c58-31c7-4513-b0e6-b737145d6528 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:12.274079 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c00c58-31c7-4513-b0e6-b737145d6528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:12.275892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c00c58-31c7-4513-b0e6-b737145d6528 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:12.293330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1381/2769] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:56:12.296118 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873d0fe-4caa-4caf-bdc1-e867f2e2af57 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:12.328757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873d0fe-4caa-4caf-bdc1-e867f2e2af57 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:12.352652 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af1d7b-2348-4ad4-9a3b-07b8c630da88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:12.353907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af1d7b-2348-4ad4-9a3b-07b8c630da88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:12.354527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af1d7b-2348-4ad4-9a3b-07b8c630da88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:12.355284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af1d7b-2348-4ad4-9a3b-07b8c630da88 admin admin] PATH_INFO: `/v3/projects/4900b525f41a4e6fa32bd08873ae7652/users/f278a8c737754e619cc3d92fefd94a02/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:12.398095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af1d7b-2348-4ad4-9a3b-07b8c630da88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4900b525f41a4e6fa32bd08873ae7652, user_id=f278a8c737754e619cc3d92fefd94a02, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:12.408694 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af1d7b-2348-4ad4-9a3b-07b8c630da88 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:12.514760 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1389/2770] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:56:12 2022] PUT /identity/v3/projects/4900b525f41a4e6fa32bd08873ae7652/users/f278a8c737754e619cc3d92fefd94a02/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:56:12.521745 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c00c58-31c7-4513-b0e6-b737145d6528 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:12.553894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c00c58-31c7-4513-b0e6-b737145d6528 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:12.705942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:12.738733 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaba6e7-7907-4cbf-85e7-e545e54829aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:12.739373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaba6e7-7907-4cbf-85e7-e545e54829aa admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:12.739649 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaba6e7-7907-4cbf-85e7-e545e54829aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:12.739917 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaba6e7-7907-4cbf-85e7-e545e54829aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:12.740654 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaba6e7-7907-4cbf-85e7-e545e54829aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:12.742965 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaba6e7-7907-4cbf-85e7-e545e54829aa admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:12.770082 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1382/2771] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:56:12.771941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af1d7b-2348-4ad4-9a3b-07b8c630da88 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:12.811124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af1d7b-2348-4ad4-9a3b-07b8c630da88 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:12.827427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c089875-29e5-495e-802f-4353a6f1dbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:12.831299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c089875-29e5-495e-802f-4353a6f1dbe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:12.836404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c089875-29e5-495e-802f-4353a6f1dbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:12.836404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c089875-29e5-495e-802f-4353a6f1dbe1 admin admin] PATH_INFO: `/v3/projects/4900b525f41a4e6fa32bd08873ae7652/users/f278a8c737754e619cc3d92fefd94a02/roles/020bc02dca71485bab2312442adaa179` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:12.837360 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c089875-29e5-495e-802f-4353a6f1dbe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4900b525f41a4e6fa32bd08873ae7652, user_id=f278a8c737754e619cc3d92fefd94a02, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:12.844681 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c089875-29e5-495e-802f-4353a6f1dbe1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:12.883517 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1390/2772] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:56:12 2022] PUT /identity/v3/projects/4900b525f41a4e6fa32bd08873ae7652/users/f278a8c737754e619cc3d92fefd94a02/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:12.894339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a634606-3513-4cea-bb98-27ef582fcbc2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:12.894865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a634606-3513-4cea-bb98-27ef582fcbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:12.895247 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a634606-3513-4cea-bb98-27ef582fcbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:12.949179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9a634606-3513-4cea-bb98-27ef582fcbc2 None None] MFA Rules not processed for user `f278a8c737754e619cc3d92fefd94a02`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:12.950597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9a634606-3513-4cea-bb98-27ef582fcbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:13.071773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1383/2773] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:13.105895 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c089875-29e5-495e-802f-4353a6f1dbe1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:13.114784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1c089875-29e5-495e-802f-4353a6f1dbe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:13.258649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c089875-29e5-495e-802f-4353a6f1dbe1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:13.287607 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c99d7-4404-4f9d-b425-fb32490a3d6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:13.290282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c99d7-4404-4f9d-b425-fb32490a3d6a service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:13.291501 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c99d7-4404-4f9d-b425-fb32490a3d6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:13.292619 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c99d7-4404-4f9d-b425-fb32490a3d6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:13.461357 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c99d7-4404-4f9d-b425-fb32490a3d6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:13.463528 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c99d7-4404-4f9d-b425-fb32490a3d6a service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:13.482611 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1391/2774] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:56:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:56:16.257514 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a634606-3513-4cea-bb98-27ef582fcbc2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:16.264534 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9a634606-3513-4cea-bb98-27ef582fcbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:16.378923 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a634606-3513-4cea-bb98-27ef582fcbc2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:16.384890 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:16.385774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:16.386084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:16.386568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] PATH_INFO: `/v3/users/708d05029cef49a4a7f7cbc78fce77ab` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:16.388132 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=708d05029cef49a4a7f7cbc78fce77ab)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:16.390288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:16.680540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '708d05029cef49a4a7f7cbc78fce77ab'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:16.701793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '708d05029cef49a4a7f7cbc78fce77ab'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:16.708993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:16.772048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '708d05029cef49a4a7f7cbc78fce77ab'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:16.774018 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '708d05029cef49a4a7f7cbc78fce77ab'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:16.785892 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '708d05029cef49a4a7f7cbc78fce77ab'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:16.795424 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1384/2775] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:56:16 2022] DELETE /identity/v3/users/708d05029cef49a4a7f7cbc78fce77ab => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:17.331951 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c99d7-4404-4f9d-b425-fb32490a3d6a service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:17.349222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c99d7-4404-4f9d-b425-fb32490a3d6a service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:17.362984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:17.440334 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:17.441278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:17.441649 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:17.442268 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] PATH_INFO: `/v3/projects/8035cc960d1f4aed981b1812e4abe6eb` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:17.444139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8035cc960d1f4aed981b1812e4abe6eb)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:17.446945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:17.654653 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8035cc960d1f4aed981b1812e4abe6eb', 'request_id': 'req-b102c61f-13ec-425b-a36c-5f6428530606'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:17.766710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8035cc960d1f4aed981b1812e4abe6eb', 'request_id': 'req-b102c61f-13ec-425b-a36c-5f6428530606'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:17.775559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8035cc960d1f4aed981b1812e4abe6eb', 'request_id': 'req-b102c61f-13ec-425b-a36c-5f6428530606'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:17.795654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] The token cache is being invalidate because project 8035cc960d1f4aed981b1812e4abe6eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:56:17.796282 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:17.797161 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:17.800140 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1392/2776] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:56:17 2022] DELETE /identity/v3/projects/8035cc960d1f4aed981b1812e4abe6eb => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:17.805276 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:17.808920 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:17.822554 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee3e28-a31f-4ffe-b7b2-1231e5f4e274 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:17.908470 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:17.909581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:17.909581 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:17.909788 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] PATH_INFO: `/v3/users/db76d309360e4403885b1a64701ae527` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:17.910871 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db76d309360e4403885b1a64701ae527)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:17.912833 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:18.147883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db76d309360e4403885b1a64701ae527'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:18.184797 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db76d309360e4403885b1a64701ae527'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:18.382457 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db76d309360e4403885b1a64701ae527'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:18.383138 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db76d309360e4403885b1a64701ae527'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:18.391436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db76d309360e4403885b1a64701ae527'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:18.402311 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1385/2777] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:56:17 2022] DELETE /identity/v3/users/db76d309360e4403885b1a64701ae527 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:56:18.808494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:18.810395 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:18.825987 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102c61f-13ec-425b-a36c-5f6428530606 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:18.901565 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e2776-bbbf-4a0d-a60b-99a89ae69eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:18.902112 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e2776-bbbf-4a0d-a60b-99a89ae69eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:18.902763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e2776-bbbf-4a0d-a60b-99a89ae69eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:18.903052 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e2776-bbbf-4a0d-a60b-99a89ae69eea admin admin] PATH_INFO: `/v3/projects/5cbdbd56714e4f1b82723835cb318fd3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:18.904872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e2776-bbbf-4a0d-a60b-99a89ae69eea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cbdbd56714e4f1b82723835cb318fd3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:18.909792 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e2776-bbbf-4a0d-a60b-99a89ae69eea admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:19.124696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:19.134460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-361e2776-bbbf-4a0d-a60b-99a89ae69eea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cbdbd56714e4f1b82723835cb318fd3', 'request_id': 'req-361e2776-bbbf-4a0d-a60b-99a89ae69eea'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:19.400568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-361e2776-bbbf-4a0d-a60b-99a89ae69eea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cbdbd56714e4f1b82723835cb318fd3', 'request_id': 'req-361e2776-bbbf-4a0d-a60b-99a89ae69eea'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:19.407156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-361e2776-bbbf-4a0d-a60b-99a89ae69eea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cbdbd56714e4f1b82723835cb318fd3', 'request_id': 'req-361e2776-bbbf-4a0d-a60b-99a89ae69eea'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:19.420226 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-361e2776-bbbf-4a0d-a60b-99a89ae69eea admin admin] The token cache is being invalidate because project 5cbdbd56714e4f1b82723835cb318fd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:56:19.420487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-361e2776-bbbf-4a0d-a60b-99a89ae69eea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:19.421191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-361e2776-bbbf-4a0d-a60b-99a89ae69eea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:19.424655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1393/2778] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:56:18 2022] DELETE /identity/v3/projects/5cbdbd56714e4f1b82723835cb318fd3 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:20.766777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:20.768297 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:20.838919 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9516e0-c17f-4486-9ea4-d9eea69d13a5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:20.862005 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:20.899817 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37110605-2211-4bd8-841c-da707b101921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:20.900419 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37110605-2211-4bd8-841c-da707b101921 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:20.900702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37110605-2211-4bd8-841c-da707b101921 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:20.900995 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37110605-2211-4bd8-841c-da707b101921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:20.902914 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-37110605-2211-4bd8-841c-da707b101921 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:21.000033 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37110605-2211-4bd8-841c-da707b101921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:21.002248 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37110605-2211-4bd8-841c-da707b101921 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:21.020250 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1386/2779] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:56:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:56:29.543494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=9,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:31.012808 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=40 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:34.216754 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc364aaa-8479-4335-80fd-02c5cfec0c26 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:34.217303 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc364aaa-8479-4335-80fd-02c5cfec0c26 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:34.217303 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc364aaa-8479-4335-80fd-02c5cfec0c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:34.237107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fc364aaa-8479-4335-80fd-02c5cfec0c26 None None] MFA Rules not processed for user `ed50f5d8261d44b487390de092ef9136`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:34.238324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fc364aaa-8479-4335-80fd-02c5cfec0c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:34.293332 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1394/2780] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:38.879303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37110605-2211-4bd8-841c-da707b101921 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:38.880499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-37110605-2211-4bd8-841c-da707b101921 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:38.953012 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37110605-2211-4bd8-841c-da707b101921 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:38.957930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fd592-e9e7-4943-87b5-8d91fe88e6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:38.958547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fd592-e9e7-4943-87b5-8d91fe88e6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:38.958825 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fd592-e9e7-4943-87b5-8d91fe88e6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:38.959095 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fd592-e9e7-4943-87b5-8d91fe88e6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:38.960845 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3f2fd592-e9e7-4943-87b5-8d91fe88e6b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:39.039911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fd592-e9e7-4943-87b5-8d91fe88e6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:39.041379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fd592-e9e7-4943-87b5-8d91fe88e6b0 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:39.054752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1387/2781] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:56:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:56:40.246082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc364aaa-8479-4335-80fd-02c5cfec0c26 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:40.247583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fc364aaa-8479-4335-80fd-02c5cfec0c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:40.261579 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc364aaa-8479-4335-80fd-02c5cfec0c26 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:40.265798 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac694d5-0fd9-486c-8110-c28c3151e375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:40.267465 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac694d5-0fd9-486c-8110-c28c3151e375 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:40.267465 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac694d5-0fd9-486c-8110-c28c3151e375 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:40.267904 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac694d5-0fd9-486c-8110-c28c3151e375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:40.269807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7ac694d5-0fd9-486c-8110-c28c3151e375 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:40.295042 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac694d5-0fd9-486c-8110-c28c3151e375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:40.296840 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac694d5-0fd9-486c-8110-c28c3151e375 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:40.313124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1395/2782] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:56:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:56:49.034659 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fd592-e9e7-4943-87b5-8d91fe88e6b0 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:49.036163 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3f2fd592-e9e7-4943-87b5-8d91fe88e6b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:49.103663 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fd592-e9e7-4943-87b5-8d91fe88e6b0 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:49.108208 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:49.108832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:49.109112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:49.109436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] PATH_INFO: `/v3/users/c0d24c3eb92a49e39462d6b2e66ba6b3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:49.111153 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0d24c3eb92a49e39462d6b2e66ba6b3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:49.113309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:49.297187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:49.300452 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0d24c3eb92a49e39462d6b2e66ba6b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:49.312819 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0d24c3eb92a49e39462d6b2e66ba6b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:49.427356 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0d24c3eb92a49e39462d6b2e66ba6b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:49.427887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0d24c3eb92a49e39462d6b2e66ba6b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:49.435256 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0d24c3eb92a49e39462d6b2e66ba6b3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:49.441055 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1388/2783] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:56:49 2022] DELETE /identity/v3/users/c0d24c3eb92a49e39462d6b2e66ba6b3 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:49.445946 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daa090-4cdc-452c-96a0-46899273bcf2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:49.446454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daa090-4cdc-452c-96a0-46899273bcf2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:49.446572 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daa090-4cdc-452c-96a0-46899273bcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:49.465783 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e5daa090-4cdc-452c-96a0-46899273bcf2 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:49.467012 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e5daa090-4cdc-452c-96a0-46899273bcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:49.526564 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1396/2784] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:49.536457 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:49.537929 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:49.597497 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3653e60-33f1-43b6-92a9-030cd1342948 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:49.601151 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae31eba-e4ad-49e7-8701-016828fa25e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:49.601748 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae31eba-e4ad-49e7-8701-016828fa25e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:49.602137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae31eba-e4ad-49e7-8701-016828fa25e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:49.602436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae31eba-e4ad-49e7-8701-016828fa25e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:49.611829 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae31eba-e4ad-49e7-8701-016828fa25e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:49.613241 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae31eba-e4ad-49e7-8701-016828fa25e3 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:49.622670 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1389/2785] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:56:50.046983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daa090-4cdc-452c-96a0-46899273bcf2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:50.048916 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e5daa090-4cdc-452c-96a0-46899273bcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:50.060829 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daa090-4cdc-452c-96a0-46899273bcf2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:50.066062 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:50.066729 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:50.067011 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:50.067276 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] PATH_INFO: `/v3/projects/b0526189598d47448cc04dd3ce93a863` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:50.068392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0526189598d47448cc04dd3ce93a863)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:50.070335 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:50.276018 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0526189598d47448cc04dd3ce93a863', 'request_id': 'req-c2d2e83c-c465-4c11-9d77-696925f9770c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:50.360139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0526189598d47448cc04dd3ce93a863', 'request_id': 'req-c2d2e83c-c465-4c11-9d77-696925f9770c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:50.366776 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0526189598d47448cc04dd3ce93a863', 'request_id': 'req-c2d2e83c-c465-4c11-9d77-696925f9770c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:50.379321 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] The token cache is being invalidate because project b0526189598d47448cc04dd3ce93a863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:56:50.379602 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:50.380245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:50.381719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1397/2786] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:56:50 2022] DELETE /identity/v3/projects/b0526189598d47448cc04dd3ce93a863 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:50.385927 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae31eba-e4ad-49e7-8701-016828fa25e3 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:50.396186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae31eba-e4ad-49e7-8701-016828fa25e3 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:50.452048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a19f5c-9aff-4759-9d60-4295d267c7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:50.452673 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a19f5c-9aff-4759-9d60-4295d267c7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:50.452947 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a19f5c-9aff-4759-9d60-4295d267c7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:50.453212 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a19f5c-9aff-4759-9d60-4295d267c7c8 admin admin] PATH_INFO: `/v3/users/f278a8c737754e619cc3d92fefd94a02` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:50.454942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a19f5c-9aff-4759-9d60-4295d267c7c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f278a8c737754e619cc3d92fefd94a02)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:50.456872 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a19f5c-9aff-4759-9d60-4295d267c7c8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:50.648601 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1a19f5c-9aff-4759-9d60-4295d267c7c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f278a8c737754e619cc3d92fefd94a02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:50.660797 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1a19f5c-9aff-4759-9d60-4295d267c7c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f278a8c737754e619cc3d92fefd94a02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:50.662544 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:50.738310 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1a19f5c-9aff-4759-9d60-4295d267c7c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f278a8c737754e619cc3d92fefd94a02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:50.738941 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1a19f5c-9aff-4759-9d60-4295d267c7c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f278a8c737754e619cc3d92fefd94a02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:50.746505 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1a19f5c-9aff-4759-9d60-4295d267c7c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f278a8c737754e619cc3d92fefd94a02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:50.752960 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1390/2787] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:56:50 2022] DELETE /identity/v3/users/f278a8c737754e619cc3d92fefd94a02 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:51.184802 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:51.186301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:51.200554 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2e83c-c465-4c11-9d77-696925f9770c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:51.213552 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:51.259671 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453d2ca-c8c1-45f6-acf6-297c616ba38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:51.260442 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453d2ca-c8c1-45f6-acf6-297c616ba38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:51.260742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453d2ca-c8c1-45f6-acf6-297c616ba38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:51.261074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453d2ca-c8c1-45f6-acf6-297c616ba38a admin admin] PATH_INFO: `/v3/projects/4900b525f41a4e6fa32bd08873ae7652` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:51.262232 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453d2ca-c8c1-45f6-acf6-297c616ba38a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4900b525f41a4e6fa32bd08873ae7652)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:51.264141 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453d2ca-c8c1-45f6-acf6-297c616ba38a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:51.429267 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8453d2ca-c8c1-45f6-acf6-297c616ba38a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4900b525f41a4e6fa32bd08873ae7652', 'request_id': 'req-8453d2ca-c8c1-45f6-acf6-297c616ba38a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:51.564692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8453d2ca-c8c1-45f6-acf6-297c616ba38a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4900b525f41a4e6fa32bd08873ae7652', 'request_id': 'req-8453d2ca-c8c1-45f6-acf6-297c616ba38a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:51.571164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8453d2ca-c8c1-45f6-acf6-297c616ba38a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4900b525f41a4e6fa32bd08873ae7652', 'request_id': 'req-8453d2ca-c8c1-45f6-acf6-297c616ba38a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:51.583236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8453d2ca-c8c1-45f6-acf6-297c616ba38a admin admin] The token cache is being invalidate because project 4900b525f41a4e6fa32bd08873ae7652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:56:51.592613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8453d2ca-c8c1-45f6-acf6-297c616ba38a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:51.592613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8453d2ca-c8c1-45f6-acf6-297c616ba38a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:51.592613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1398/2788] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:56:51 2022] DELETE /identity/v3/projects/4900b525f41a4e6fa32bd08873ae7652 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:51.592991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0b5e0-394e-4fb5-9008-9aa2f378cf9b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:51.592991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0b5e0-394e-4fb5-9008-9aa2f378cf9b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:51.592991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0b5e0-394e-4fb5-9008-9aa2f378cf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:51.615915 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-27c0b5e0-394e-4fb5-9008-9aa2f378cf9b None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:51.617147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-27c0b5e0-394e-4fb5-9008-9aa2f378cf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:51.676511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1391/2789] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:51.684572 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56492a-ac39-4454-a6bf-87070a2b6c0e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:51.684781 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56492a-ac39-4454-a6bf-87070a2b6c0e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:51.685058 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56492a-ac39-4454-a6bf-87070a2b6c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:51.710644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6d56492a-ac39-4454-a6bf-87070a2b6c0e None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:51.711874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6d56492a-ac39-4454-a6bf-87070a2b6c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:51.719899 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1399/2790] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:51.724937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0b5e0-394e-4fb5-9008-9aa2f378cf9b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:51.734874 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0b5e0-394e-4fb5-9008-9aa2f378cf9b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:51.739967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae71fd3-cc8e-4c11-8607-4683c9568637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:51.740566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae71fd3-cc8e-4c11-8607-4683c9568637 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:51.740839 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae71fd3-cc8e-4c11-8607-4683c9568637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:51.741105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae71fd3-cc8e-4c11-8607-4683c9568637 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:51.741749 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae71fd3-cc8e-4c11-8607-4683c9568637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:56:51.742043 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae71fd3-cc8e-4c11-8607-4683c9568637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:51.743652 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae71fd3-cc8e-4c11-8607-4683c9568637 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:51.756521 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1392/2791] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:56:51.761121 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56492a-ac39-4454-a6bf-87070a2b6c0e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:51.771187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56492a-ac39-4454-a6bf-87070a2b6c0e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:51.775995 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b1810-7bf4-4012-8c25-30160873fb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:51.776583 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b1810-7bf4-4012-8c25-30160873fb78 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:51.776858 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b1810-7bf4-4012-8c25-30160873fb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:51.777126 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b1810-7bf4-4012-8c25-30160873fb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:51.777795 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b1810-7bf4-4012-8c25-30160873fb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:51.779733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b1810-7bf4-4012-8c25-30160873fb78 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:51.885455 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1400/2792] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:56:51 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:56:51.886704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae71fd3-cc8e-4c11-8607-4683c9568637 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:51.897503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae71fd3-cc8e-4c11-8607-4683c9568637 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:51.954047 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a8b7d-4a72-4105-bfdd-0a96909f02a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:51.955163 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a8b7d-4a72-4105-bfdd-0a96909f02a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:51.955441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a8b7d-4a72-4105-bfdd-0a96909f02a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:51.955732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a8b7d-4a72-4105-bfdd-0a96909f02a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:51.956338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a8b7d-4a72-4105-bfdd-0a96909f02a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:51.958263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a8b7d-4a72-4105-bfdd-0a96909f02a6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:52.196540 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1393/2793] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:56:51 2022] POST /identity/v3/users => generated 291 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:56:52.201138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b1810-7bf4-4012-8c25-30160873fb78 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:52.212878 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b1810-7bf4-4012-8c25-30160873fb78 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:52.217802 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdb7c6-8410-4bbb-89dd-cf8ed3c7b6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:52.218422 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdb7c6-8410-4bbb-89dd-cf8ed3c7b6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:52.218696 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdb7c6-8410-4bbb-89dd-cf8ed3c7b6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:52.218962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cdb7c6-8410-4bbb-89dd-cf8ed3c7b6e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:52.219593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdb7c6-8410-4bbb-89dd-cf8ed3c7b6e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:52.221997 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cdb7c6-8410-4bbb-89dd-cf8ed3c7b6e8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:52.232463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1401/2794] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:56:52.237230 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a8b7d-4a72-4105-bfdd-0a96909f02a6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:52.247570 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a8b7d-4a72-4105-bfdd-0a96909f02a6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:52.252265 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6d14e-82d6-44ea-a302-e24e118d6851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:52.252856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6d14e-82d6-44ea-a302-e24e118d6851 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:52.253128 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6d14e-82d6-44ea-a302-e24e118d6851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:52.253392 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6d14e-82d6-44ea-a302-e24e118d6851 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:52.254021 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6d14e-82d6-44ea-a302-e24e118d6851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:52.255630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6d14e-82d6-44ea-a302-e24e118d6851 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:52.267986 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1394/2795] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:56:52.277359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdb7c6-8410-4bbb-89dd-cf8ed3c7b6e8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:52.289491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cdb7c6-8410-4bbb-89dd-cf8ed3c7b6e8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:52.295403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91bd2c8-61c5-4b70-a16a-ea2dfa5e58ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:52.300346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bd2c8-61c5-4b70-a16a-ea2dfa5e58ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:52.300346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bd2c8-61c5-4b70-a16a-ea2dfa5e58ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:52.300346 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bd2c8-61c5-4b70-a16a-ea2dfa5e58ab admin admin] PATH_INFO: `/v3/projects/9266a99938c54a73988a3cd3a7514a80/users/7916a0b1dacd410e9ae25c58d6904ce6/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:52.312328 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91bd2c8-61c5-4b70-a16a-ea2dfa5e58ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=9266a99938c54a73988a3cd3a7514a80, user_id=7916a0b1dacd410e9ae25c58d6904ce6, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:52.315617 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91bd2c8-61c5-4b70-a16a-ea2dfa5e58ab admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:52.458014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1402/2796] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:56:52 2022] PUT /identity/v3/projects/9266a99938c54a73988a3cd3a7514a80/users/7916a0b1dacd410e9ae25c58d6904ce6/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:52.475343 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472eb3c2-a85b-4c7c-b42b-1487ea8ded9c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:52.475583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472eb3c2-a85b-4c7c-b42b-1487ea8ded9c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:52.475888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472eb3c2-a85b-4c7c-b42b-1487ea8ded9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:52.505643 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-472eb3c2-a85b-4c7c-b42b-1487ea8ded9c None None] MFA Rules not processed for user `7916a0b1dacd410e9ae25c58d6904ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:52.507007 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-472eb3c2-a85b-4c7c-b42b-1487ea8ded9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:52.547215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:52.576153 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1395/2797] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:52.581305 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91bd2c8-61c5-4b70-a16a-ea2dfa5e58ab admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:52.623074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:52.657990 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91bd2c8-61c5-4b70-a16a-ea2dfa5e58ab admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:52.662882 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed0300-2ea3-4884-85e8-fc611e52ac99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:52.663520 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed0300-2ea3-4884-85e8-fc611e52ac99 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:52.664022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed0300-2ea3-4884-85e8-fc611e52ac99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:52.664300 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed0300-2ea3-4884-85e8-fc611e52ac99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:52.664943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed0300-2ea3-4884-85e8-fc611e52ac99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:52.667090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed0300-2ea3-4884-85e8-fc611e52ac99 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:52.756562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1403/2798] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:56:52 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:56:52.760615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472eb3c2-a85b-4c7c-b42b-1487ea8ded9c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:52.772030 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472eb3c2-a85b-4c7c-b42b-1487ea8ded9c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:52.827881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5209a4-389c-4c6e-be2a-a9311cb57d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:52.828504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5209a4-389c-4c6e-be2a-a9311cb57d98 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:52.828775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5209a4-389c-4c6e-be2a-a9311cb57d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:52.829047 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5209a4-389c-4c6e-be2a-a9311cb57d98 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:52.830366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5209a4-389c-4c6e-be2a-a9311cb57d98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:52.832288 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5209a4-389c-4c6e-be2a-a9311cb57d98 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:52.960596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1396/2799] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:56:52 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:56:52.964868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed0300-2ea3-4884-85e8-fc611e52ac99 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:52.975108 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed0300-2ea3-4884-85e8-fc611e52ac99 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:52.981148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef0999-43ad-4cb3-a8c5-a8a6a60db989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:52.981148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef0999-43ad-4cb3-a8c5-a8a6a60db989 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:52.981148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef0999-43ad-4cb3-a8c5-a8a6a60db989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:52.981148 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef0999-43ad-4cb3-a8c5-a8a6a60db989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:52.981573 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef0999-43ad-4cb3-a8c5-a8a6a60db989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:52.982827 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef0999-43ad-4cb3-a8c5-a8a6a60db989 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:52.993278 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1404/2800] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:56:52.997515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5209a4-389c-4c6e-be2a-a9311cb57d98 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:53.008366 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5209a4-389c-4c6e-be2a-a9311cb57d98 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:53.012918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573b8b7-69eb-4752-b16a-b8f6013dae8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:53.013547 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573b8b7-69eb-4752-b16a-b8f6013dae8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:53.013823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573b8b7-69eb-4752-b16a-b8f6013dae8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:53.014091 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573b8b7-69eb-4752-b16a-b8f6013dae8b admin admin] PATH_INFO: `/v3/projects/3e0e60b7050a44209563691282163d0f/users/5ae4bddaedce42bd94435b65ae28ccbb/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:53.029069 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573b8b7-69eb-4752-b16a-b8f6013dae8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0e60b7050a44209563691282163d0f, user_id=5ae4bddaedce42bd94435b65ae28ccbb, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:53.031774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573b8b7-69eb-4752-b16a-b8f6013dae8b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:53.102832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1397/2801] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:56:52 2022] PUT /identity/v3/projects/3e0e60b7050a44209563691282163d0f/users/5ae4bddaedce42bd94435b65ae28ccbb/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:56:53.106880 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef0999-43ad-4cb3-a8c5-a8a6a60db989 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:53.117720 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef0999-43ad-4cb3-a8c5-a8a6a60db989 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:53.174727 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e0246-922d-4b7c-baf7-67fe557d5a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:53.175317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0246-922d-4b7c-baf7-67fe557d5a6d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:53.175590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0246-922d-4b7c-baf7-67fe557d5a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:53.175853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e0246-922d-4b7c-baf7-67fe557d5a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:53.176483 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e0246-922d-4b7c-baf7-67fe557d5a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:53.178084 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e0246-922d-4b7c-baf7-67fe557d5a6d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:53.187704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1405/2802] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:56:53.192600 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573b8b7-69eb-4752-b16a-b8f6013dae8b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:53.203046 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573b8b7-69eb-4752-b16a-b8f6013dae8b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:53.207755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2551a43-8243-4746-b1e1-ddb556d243a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:53.208406 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2551a43-8243-4746-b1e1-ddb556d243a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:53.208681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2551a43-8243-4746-b1e1-ddb556d243a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:53.208950 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2551a43-8243-4746-b1e1-ddb556d243a9 admin admin] PATH_INFO: `/v3/projects/3e0e60b7050a44209563691282163d0f/users/5ae4bddaedce42bd94435b65ae28ccbb/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:53.210958 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2551a43-8243-4746-b1e1-ddb556d243a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0e60b7050a44209563691282163d0f, user_id=5ae4bddaedce42bd94435b65ae28ccbb, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:53.213570 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2551a43-8243-4746-b1e1-ddb556d243a9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:53.224703 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1398/2803] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:56:53 2022] PUT /identity/v3/projects/3e0e60b7050a44209563691282163d0f/users/5ae4bddaedce42bd94435b65ae28ccbb/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:53.236874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b6db5-6bb0-459a-a4e5-be04cc0e160d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:53.237451 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b6db5-6bb0-459a-a4e5-be04cc0e160d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:53.237451 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b6db5-6bb0-459a-a4e5-be04cc0e160d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:53.265365 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-103b6db5-6bb0-459a-a4e5-be04cc0e160d None None] MFA Rules not processed for user `5ae4bddaedce42bd94435b65ae28ccbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:53.266531 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-103b6db5-6bb0-459a-a4e5-be04cc0e160d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:53.340819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1406/2804] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:53.352529 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2551a43-8243-4746-b1e1-ddb556d243a9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:53.354287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e2551a43-8243-4746-b1e1-ddb556d243a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:53.412060 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2551a43-8243-4746-b1e1-ddb556d243a9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:53.415832 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40ce8a-8f56-4d9d-a64b-944bd16f32f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:53.416427 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40ce8a-8f56-4d9d-a64b-944bd16f32f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:53.416701 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40ce8a-8f56-4d9d-a64b-944bd16f32f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:53.416967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40ce8a-8f56-4d9d-a64b-944bd16f32f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:53.426637 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40ce8a-8f56-4d9d-a64b-944bd16f32f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:53.428732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40ce8a-8f56-4d9d-a64b-944bd16f32f8 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:53.437450 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1399/2805] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:56:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:56:53.980329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b6db5-6bb0-459a-a4e5-be04cc0e160d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:54.011434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:54.043407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b6db5-6bb0-459a-a4e5-be04cc0e160d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:54.048485 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:54.049114 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:54.049389 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:54.050263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] PATH_INFO: `/v3/users/7916a0b1dacd410e9ae25c58d6904ce6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:54.050966 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7916a0b1dacd410e9ae25c58d6904ce6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:54.052983 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:54.305684 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7916a0b1dacd410e9ae25c58d6904ce6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:54.306371 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7916a0b1dacd410e9ae25c58d6904ce6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:54.434131 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7916a0b1dacd410e9ae25c58d6904ce6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:54.442642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7916a0b1dacd410e9ae25c58d6904ce6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:54.456466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7916a0b1dacd410e9ae25c58d6904ce6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:54.462512 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1407/2806] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:56:53 2022] DELETE /identity/v3/users/7916a0b1dacd410e9ae25c58d6904ce6 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:54.467264 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fbbff-c2d1-41a4-ba08-5ceacf9a5212 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:54.467785 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fbbff-c2d1-41a4-ba08-5ceacf9a5212 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:54.467895 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fbbff-c2d1-41a4-ba08-5ceacf9a5212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:54.487203 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a78fbbff-c2d1-41a4-ba08-5ceacf9a5212 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:54.488420 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a78fbbff-c2d1-41a4-ba08-5ceacf9a5212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:54.519917 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:54.550169 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1400/2807] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:54.568477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:54.568477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:54.621310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455876-77db-48cc-a0b9-9800997e4f39 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:54.625001 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0ad5d-3f27-46e4-86ac-f6a575035e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:54.625659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0ad5d-3f27-46e4-86ac-f6a575035e5e service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:54.626744 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0ad5d-3f27-46e4-86ac-f6a575035e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:54.627024 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0ad5d-3f27-46e4-86ac-f6a575035e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:54.636265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0ad5d-3f27-46e4-86ac-f6a575035e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:54.637692 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0ad5d-3f27-46e4-86ac-f6a575035e5e service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:54.648030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1408/2808] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:56:55.201860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fbbff-c2d1-41a4-ba08-5ceacf9a5212 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:55.203444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a78fbbff-c2d1-41a4-ba08-5ceacf9a5212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:55.217444 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fbbff-c2d1-41a4-ba08-5ceacf9a5212 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:55.222775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:55.223412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:55.223680 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:55.223947 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] PATH_INFO: `/v3/projects/9266a99938c54a73988a3cd3a7514a80` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:55.225124 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9266a99938c54a73988a3cd3a7514a80)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:55.227244 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:55.344014 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9266a99938c54a73988a3cd3a7514a80', 'request_id': 'req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:55.456814 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9266a99938c54a73988a3cd3a7514a80', 'request_id': 'req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:55.462876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9266a99938c54a73988a3cd3a7514a80', 'request_id': 'req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:55.474515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] The token cache is being invalidate because project 9266a99938c54a73988a3cd3a7514a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:56:55.475126 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:55.475412 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:55.476911 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1401/2809] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:56:55 2022] DELETE /identity/v3/projects/9266a99938c54a73988a3cd3a7514a80 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:55.480931 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0ad5d-3f27-46e4-86ac-f6a575035e5e service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:55.491127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0ad5d-3f27-46e4-86ac-f6a575035e5e service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:55.538447 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:55.548156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36738acb-4c24-4de9-bb9f-9975e35f0c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:55.548769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738acb-4c24-4de9-bb9f-9975e35f0c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:55.549065 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738acb-4c24-4de9-bb9f-9975e35f0c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:55.549333 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738acb-4c24-4de9-bb9f-9975e35f0c48 admin admin] PATH_INFO: `/v3/users/5ae4bddaedce42bd94435b65ae28ccbb` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:55.550461 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36738acb-4c24-4de9-bb9f-9975e35f0c48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ae4bddaedce42bd94435b65ae28ccbb)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:55.552444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36738acb-4c24-4de9-bb9f-9975e35f0c48 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:55.727986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-36738acb-4c24-4de9-bb9f-9975e35f0c48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ae4bddaedce42bd94435b65ae28ccbb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:55.728557 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-36738acb-4c24-4de9-bb9f-9975e35f0c48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ae4bddaedce42bd94435b65ae28ccbb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:55.793113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-36738acb-4c24-4de9-bb9f-9975e35f0c48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ae4bddaedce42bd94435b65ae28ccbb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:55.796571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-36738acb-4c24-4de9-bb9f-9975e35f0c48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ae4bddaedce42bd94435b65ae28ccbb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:55.808659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-36738acb-4c24-4de9-bb9f-9975e35f0c48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ae4bddaedce42bd94435b65ae28ccbb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:55.815080 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1409/2810] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:56:55 2022] DELETE /identity/v3/users/5ae4bddaedce42bd94435b65ae28ccbb => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:56.527316 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:56.528811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:56.543176 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd2f4c-0c21-4358-af0f-21ccca5fdba5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:56.601726 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b96d2f-a965-4a19-8313-f4998d389f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:56.602389 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b96d2f-a965-4a19-8313-f4998d389f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:56.602665 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b96d2f-a965-4a19-8313-f4998d389f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:56.603066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b96d2f-a965-4a19-8313-f4998d389f05 admin admin] PATH_INFO: `/v3/projects/3e0e60b7050a44209563691282163d0f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:56.604224 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b96d2f-a965-4a19-8313-f4998d389f05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e0e60b7050a44209563691282163d0f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:56.606109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b96d2f-a965-4a19-8313-f4998d389f05 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:56.711776 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-04b96d2f-a965-4a19-8313-f4998d389f05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e0e60b7050a44209563691282163d0f', 'request_id': 'req-04b96d2f-a965-4a19-8313-f4998d389f05'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:56.804702 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-04b96d2f-a965-4a19-8313-f4998d389f05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e0e60b7050a44209563691282163d0f', 'request_id': 'req-04b96d2f-a965-4a19-8313-f4998d389f05'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:56.807186 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:56.811338 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-04b96d2f-a965-4a19-8313-f4998d389f05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e0e60b7050a44209563691282163d0f', 'request_id': 'req-04b96d2f-a965-4a19-8313-f4998d389f05'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:56.823321 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-04b96d2f-a965-4a19-8313-f4998d389f05 admin admin] The token cache is being invalidate because project 3e0e60b7050a44209563691282163d0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:56:56.823608 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-04b96d2f-a965-4a19-8313-f4998d389f05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:56.824248 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-04b96d2f-a965-4a19-8313-f4998d389f05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:56:56.825844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1402/2811] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:56:56 2022] DELETE /identity/v3/projects/3e0e60b7050a44209563691282163d0f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:56.830830 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f80c3-5cf1-414e-9952-bcac6b30ec58 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:56.831330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f80c3-5cf1-414e-9952-bcac6b30ec58 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:56.831446 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f80c3-5cf1-414e-9952-bcac6b30ec58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:56.852353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9f2f80c3-5cf1-414e-9952-bcac6b30ec58 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:56.859329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9f2f80c3-5cf1-414e-9952-bcac6b30ec58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:56.913200 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1410/2812] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:56.921395 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318b47b-e1f3-44ee-9869-ede3be07c65c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:56.921678 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318b47b-e1f3-44ee-9869-ede3be07c65c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:56.921952 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318b47b-e1f3-44ee-9869-ede3be07c65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:56.943823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d318b47b-e1f3-44ee-9869-ede3be07c65c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:56.945028 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d318b47b-e1f3-44ee-9869-ede3be07c65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:56.953774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1403/2813] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:56.958313 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f80c3-5cf1-414e-9952-bcac6b30ec58 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:56.968045 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f80c3-5cf1-414e-9952-bcac6b30ec58 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:56.973102 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe794c-ff6e-4ece-b97a-c501554144f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:56.973723 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe794c-ff6e-4ece-b97a-c501554144f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:56.973999 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe794c-ff6e-4ece-b97a-c501554144f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:56.974304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe794c-ff6e-4ece-b97a-c501554144f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:56.974940 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe794c-ff6e-4ece-b97a-c501554144f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:56:56.975237 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe794c-ff6e-4ece-b97a-c501554144f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:56.976820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe794c-ff6e-4ece-b97a-c501554144f2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:56.988854 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1411/2814] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 08:56:56.993248 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318b47b-e1f3-44ee-9869-ede3be07c65c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:57.002546 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318b47b-e1f3-44ee-9869-ede3be07c65c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:57.007133 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727dadf4-7348-4183-8c6e-9eb553d00398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:57.007713 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dadf4-7348-4183-8c6e-9eb553d00398 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:57.007984 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dadf4-7348-4183-8c6e-9eb553d00398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:57.008248 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dadf4-7348-4183-8c6e-9eb553d00398 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:57.008872 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727dadf4-7348-4183-8c6e-9eb553d00398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:57.010770 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727dadf4-7348-4183-8c6e-9eb553d00398 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:57.098742 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1404/2815] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:56:56 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:56:57.102957 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe794c-ff6e-4ece-b97a-c501554144f2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:57.114434 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe794c-ff6e-4ece-b97a-c501554144f2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:57.165874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:56:57.172001 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3042a0-bd7d-4823-b0e2-d401c2200f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:57.178868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3042a0-bd7d-4823-b0e2-d401c2200f99 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:57.178868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3042a0-bd7d-4823-b0e2-d401c2200f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:57.178868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3042a0-bd7d-4823-b0e2-d401c2200f99 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:57.178868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3042a0-bd7d-4823-b0e2-d401c2200f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:57.178868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3042a0-bd7d-4823-b0e2-d401c2200f99 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:57.263292 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1412/2816] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:56:57 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:56:57.268089 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727dadf4-7348-4183-8c6e-9eb553d00398 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:57.278516 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727dadf4-7348-4183-8c6e-9eb553d00398 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:57.283151 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177c30a-e3f0-463b-bbf8-ed081b4ee517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:57.284266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177c30a-e3f0-463b-bbf8-ed081b4ee517 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:57.284555 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177c30a-e3f0-463b-bbf8-ed081b4ee517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:57.284823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177c30a-e3f0-463b-bbf8-ed081b4ee517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:57.285455 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177c30a-e3f0-463b-bbf8-ed081b4ee517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:57.287073 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177c30a-e3f0-463b-bbf8-ed081b4ee517 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:57.297793 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1405/2817] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:56:57.303109 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3042a0-bd7d-4823-b0e2-d401c2200f99 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:57.313379 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3042a0-bd7d-4823-b0e2-d401c2200f99 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:57.320425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4de35-6827-48e4-85e1-bb1d22d32842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:57.320555 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4de35-6827-48e4-85e1-bb1d22d32842 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:57.321014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4de35-6827-48e4-85e1-bb1d22d32842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:57.321301 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4de35-6827-48e4-85e1-bb1d22d32842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:57.321962 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4de35-6827-48e4-85e1-bb1d22d32842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:57.323636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4de35-6827-48e4-85e1-bb1d22d32842 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:57.335590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1413/2818] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:56:57.339584 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177c30a-e3f0-463b-bbf8-ed081b4ee517 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:57.351276 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177c30a-e3f0-463b-bbf8-ed081b4ee517 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:57.356199 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdaf7d-9215-43ad-bf7c-ad92597d7fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:57.356910 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdaf7d-9215-43ad-bf7c-ad92597d7fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:57.357187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdaf7d-9215-43ad-bf7c-ad92597d7fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:57.357459 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfdaf7d-9215-43ad-bf7c-ad92597d7fc3 admin admin] PATH_INFO: `/v3/projects/dfefe384ec9e490cadc95e22e88b5a9a/users/7a5740aecbba4893814cc45742762f97/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:57.373628 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfdaf7d-9215-43ad-bf7c-ad92597d7fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfefe384ec9e490cadc95e22e88b5a9a, user_id=7a5740aecbba4893814cc45742762f97, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:57.376398 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfdaf7d-9215-43ad-bf7c-ad92597d7fc3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:57.451809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1406/2819] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:56:57 2022] PUT /identity/v3/projects/dfefe384ec9e490cadc95e22e88b5a9a/users/7a5740aecbba4893814cc45742762f97/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:56:57.519410 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076b746-aeeb-491b-9d66-282bd1adb49c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:57.519688 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076b746-aeeb-491b-9d66-282bd1adb49c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:57.519956 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076b746-aeeb-491b-9d66-282bd1adb49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:57.548099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2076b746-aeeb-491b-9d66-282bd1adb49c None None] MFA Rules not processed for user `7a5740aecbba4893814cc45742762f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:56:57.549314 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2076b746-aeeb-491b-9d66-282bd1adb49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:57.617896 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1414/2820] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:56:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:56:57.629275 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdaf7d-9215-43ad-bf7c-ad92597d7fc3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:57.631519 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-abfdaf7d-9215-43ad-bf7c-ad92597d7fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:57.687789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfdaf7d-9215-43ad-bf7c-ad92597d7fc3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:57.692232 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9969b-6316-4863-836e-15b9f3292182 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b86978df61245ae88fead154193fcd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:57.692840 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9969b-6316-4863-836e-15b9f3292182 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:57.693120 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9969b-6316-4863-836e-15b9f3292182 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:57.693379 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9969b-6316-4863-836e-15b9f3292182 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:57.702045 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c9969b-6316-4863-836e-15b9f3292182 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:57.703500 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c9969b-6316-4863-836e-15b9f3292182 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:57.713044 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1407/2821] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:56:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:56:58.859114 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076b746-aeeb-491b-9d66-282bd1adb49c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:56:58.860635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2076b746-aeeb-491b-9d66-282bd1adb49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:58.917907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076b746-aeeb-491b-9d66-282bd1adb49c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:56:58.921572 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74136e5b-1b73-463f-8c69-f6db894bcd85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:56:58.922188 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74136e5b-1b73-463f-8c69-f6db894bcd85 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:56:58.922467 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74136e5b-1b73-463f-8c69-f6db894bcd85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:56:58.922728 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74136e5b-1b73-463f-8c69-f6db894bcd85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:56:58.924426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-74136e5b-1b73-463f-8c69-f6db894bcd85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:56:58.987311 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74136e5b-1b73-463f-8c69-f6db894bcd85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:56:58.988763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74136e5b-1b73-463f-8c69-f6db894bcd85 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:56:58.998263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1415/2822] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:56:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:57:07.708937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=58,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:57:08.994077 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:57:16.174865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9969b-6316-4863-836e-15b9f3292182 service glance] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:16.237436 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9969b-6316-4863-836e-15b9f3292182 service glance] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:16.241870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:16.242522 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:16.242798 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:16.243062 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] PATH_INFO: `/v3/users/7a5740aecbba4893814cc45742762f97` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:16.244167 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5740aecbba4893814cc45742762f97)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:16.246042 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:16.409223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5740aecbba4893814cc45742762f97'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:16.421752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5740aecbba4893814cc45742762f97'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:16.473816 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5740aecbba4893814cc45742762f97'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:16.474520 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5740aecbba4893814cc45742762f97'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:16.482194 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5740aecbba4893814cc45742762f97'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:16.488507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1408/2823] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:57:16 2022] DELETE /identity/v3/users/7a5740aecbba4893814cc45742762f97 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:57:16.493255 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2daea7-6c5e-4cbb-bf0b-69e61a5c8f44 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:16.493759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2daea7-6c5e-4cbb-bf0b-69e61a5c8f44 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:16.493872 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2daea7-6c5e-4cbb-bf0b-69e61a5c8f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:16.514902 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6d2daea7-6c5e-4cbb-bf0b-69e61a5c8f44 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:57:16.516152 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6d2daea7-6c5e-4cbb-bf0b-69e61a5c8f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:16.587857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1416/2824] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:57:16.599156 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:16.600629 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:16.666431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186d391-8857-4b70-8623-a3f81c7deebd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:16.672571 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdde1af-7d6f-4976-bf6d-c2858be4390c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:16.673520 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdde1af-7d6f-4976-bf6d-c2858be4390c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:16.673985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdde1af-7d6f-4976-bf6d-c2858be4390c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:16.674513 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdde1af-7d6f-4976-bf6d-c2858be4390c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:16.684786 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdde1af-7d6f-4976-bf6d-c2858be4390c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:16.686346 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdde1af-7d6f-4976-bf6d-c2858be4390c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:16.697422 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1409/2825] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:57:17.146691 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2daea7-6c5e-4cbb-bf0b-69e61a5c8f44 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:17.148392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6d2daea7-6c5e-4cbb-bf0b-69e61a5c8f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:17.162304 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2daea7-6c5e-4cbb-bf0b-69e61a5c8f44 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:17.168491 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:17.169261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:17.169675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:17.170009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62 admin admin] PATH_INFO: `/v3/projects/dfefe384ec9e490cadc95e22e88b5a9a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:17.171344 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfefe384ec9e490cadc95e22e88b5a9a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:17.173526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:17.319519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfefe384ec9e490cadc95e22e88b5a9a', 'request_id': 'req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:17.393739 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfefe384ec9e490cadc95e22e88b5a9a', 'request_id': 'req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:17.401522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfefe384ec9e490cadc95e22e88b5a9a', 'request_id': 'req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:17.416377 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62 admin admin] The token cache is being invalidate because project dfefe384ec9e490cadc95e22e88b5a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:57:17.416701 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:17.417370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9fe1ada6-5ac1-4e8e-af3a-ce255cea0e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:17.418919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1417/2826] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:57:17 2022] DELETE /identity/v3/projects/dfefe384ec9e490cadc95e22e88b5a9a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:57:17.424092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3a146-fb6f-4eb9-b317-d71ec7b9960c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:17.424580 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3a146-fb6f-4eb9-b317-d71ec7b9960c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:17.424699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3a146-fb6f-4eb9-b317-d71ec7b9960c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:17.445943 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d7b3a146-fb6f-4eb9-b317-d71ec7b9960c None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:57:17.447105 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d7b3a146-fb6f-4eb9-b317-d71ec7b9960c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:17.465674 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:57:17.510056 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1410/2827] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:57:17.517907 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9191f43-d4f1-44c2-8bf9-19ca890ddaa5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:17.518392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9191f43-d4f1-44c2-8bf9-19ca890ddaa5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:17.518392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9191f43-d4f1-44c2-8bf9-19ca890ddaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:17.541206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e9191f43-d4f1-44c2-8bf9-19ca890ddaa5 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:57:17.542778 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e9191f43-d4f1-44c2-8bf9-19ca890ddaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:17.551183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1418/2828] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:57:17.555301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3a146-fb6f-4eb9-b317-d71ec7b9960c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:17.565367 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3a146-fb6f-4eb9-b317-d71ec7b9960c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:17.570266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6a000-7868-4ddc-b1b1-0754c296bd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:17.570876 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6a000-7868-4ddc-b1b1-0754c296bd51 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:17.571150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6a000-7868-4ddc-b1b1-0754c296bd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:17.571415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6a000-7868-4ddc-b1b1-0754c296bd51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:17.572122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6a000-7868-4ddc-b1b1-0754c296bd51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:57:17.572369 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6a000-7868-4ddc-b1b1-0754c296bd51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:17.573959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6a000-7868-4ddc-b1b1-0754c296bd51 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:17.591815 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9191f43-d4f1-44c2-8bf9-19ca890ddaa5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:17.596730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1411/2829] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:57:17.602335 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9191f43-d4f1-44c2-8bf9-19ca890ddaa5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:17.607249 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96f1ad-1786-4784-9f9e-2d7825631ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:17.607836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96f1ad-1786-4784-9f9e-2d7825631ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:17.608132 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96f1ad-1786-4784-9f9e-2d7825631ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:17.608402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96f1ad-1786-4784-9f9e-2d7825631ca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:17.609040 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96f1ad-1786-4784-9f9e-2d7825631ca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:17.610982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96f1ad-1786-4784-9f9e-2d7825631ca0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:17.774551 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1419/2830] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:57:17 2022] POST /identity/v3/projects => generated 371 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:57:17.779298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6a000-7868-4ddc-b1b1-0754c296bd51 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:17.797127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6a000-7868-4ddc-b1b1-0754c296bd51 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:17.869093 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6bad7-eb93-465c-8cac-b29d9bdadddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:17.870020 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6bad7-eb93-465c-8cac-b29d9bdadddc admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:17.870340 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6bad7-eb93-465c-8cac-b29d9bdadddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:17.871108 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6bad7-eb93-465c-8cac-b29d9bdadddc admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:17.871238 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6bad7-eb93-465c-8cac-b29d9bdadddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:17.873215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6bad7-eb93-465c-8cac-b29d9bdadddc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:17.984870 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1412/2831] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:57:17 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:57:17.990113 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96f1ad-1786-4784-9f9e-2d7825631ca0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:18.001777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96f1ad-1786-4784-9f9e-2d7825631ca0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:18.008609 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20d95a-c207-4ef0-809c-5cbee4a0ff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:18.009235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20d95a-c207-4ef0-809c-5cbee4a0ff30 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:18.009590 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20d95a-c207-4ef0-809c-5cbee4a0ff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:18.009861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20d95a-c207-4ef0-809c-5cbee4a0ff30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:18.010548 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20d95a-c207-4ef0-809c-5cbee4a0ff30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:18.012806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20d95a-c207-4ef0-809c-5cbee4a0ff30 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:18.024333 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1420/2832] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:57:18.028239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6bad7-eb93-465c-8cac-b29d9bdadddc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:18.038768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6bad7-eb93-465c-8cac-b29d9bdadddc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:18.043650 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c2c99-3022-474c-bbc0-b789e9a2c62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:18.044188 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c2c99-3022-474c-bbc0-b789e9a2c62b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:18.044526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c2c99-3022-474c-bbc0-b789e9a2c62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:18.044795 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c2c99-3022-474c-bbc0-b789e9a2c62b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:18.045447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c2c99-3022-474c-bbc0-b789e9a2c62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:18.047101 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c2c99-3022-474c-bbc0-b789e9a2c62b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:18.058883 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1413/2833] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:57:18.063464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20d95a-c207-4ef0-809c-5cbee4a0ff30 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:18.074789 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20d95a-c207-4ef0-809c-5cbee4a0ff30 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:18.079480 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e9837-9171-46ff-9897-e436f30a260d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:18.080144 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e9837-9171-46ff-9897-e436f30a260d admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:18.081012 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e9837-9171-46ff-9897-e436f30a260d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:18.081366 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e9837-9171-46ff-9897-e436f30a260d admin admin] PATH_INFO: `/v3/projects/9cb7ffe0a3b842d4947ff92e84d082cd/users/2275e341a8f94b0e9189832ef61cb744/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:18.100212 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e9837-9171-46ff-9897-e436f30a260d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb7ffe0a3b842d4947ff92e84d082cd, user_id=2275e341a8f94b0e9189832ef61cb744, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:18.103806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e9837-9171-46ff-9897-e436f30a260d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:18.172944 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1421/2834] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:57:18 2022] PUT /identity/v3/projects/9cb7ffe0a3b842d4947ff92e84d082cd/users/2275e341a8f94b0e9189832ef61cb744/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:57:18.183712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824e2e7-2414-4ea7-9aad-7acb03663c10 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:18.184077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824e2e7-2414-4ea7-9aad-7acb03663c10 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:18.184263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824e2e7-2414-4ea7-9aad-7acb03663c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:18.218814 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c824e2e7-2414-4ea7-9aad-7acb03663c10 None None] MFA Rules not processed for user `2275e341a8f94b0e9189832ef61cb744`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:57:18.220031 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c824e2e7-2414-4ea7-9aad-7acb03663c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:18.309245 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1414/2835] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:57:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:57:18.323198 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e9837-9171-46ff-9897-e436f30a260d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:18.325441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-635e9837-9171-46ff-9897-e436f30a260d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:18.352139 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:57:18.396484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e9837-9171-46ff-9897-e436f30a260d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:18.400383 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f33dc3-57e4-41da-8763-227af6d535b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:18.401018 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f33dc3-57e4-41da-8763-227af6d535b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:18.401293 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f33dc3-57e4-41da-8763-227af6d535b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:18.401562 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f33dc3-57e4-41da-8763-227af6d535b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:18.415169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f33dc3-57e4-41da-8763-227af6d535b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:18.415169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f33dc3-57e4-41da-8763-227af6d535b1 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:18.424054 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1422/2836] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:57:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:57:18.452394 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824e2e7-2414-4ea7-9aad-7acb03663c10 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:18.483904 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:57:18.549397 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824e2e7-2414-4ea7-9aad-7acb03663c10 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:18.557877 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:18.558567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:18.559330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:18.559330 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] PATH_INFO: `/v3/users/2275e341a8f94b0e9189832ef61cb744` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:18.560531 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2275e341a8f94b0e9189832ef61cb744)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:18.562499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:18.767501 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2275e341a8f94b0e9189832ef61cb744'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:18.779994 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2275e341a8f94b0e9189832ef61cb744'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:18.879566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2275e341a8f94b0e9189832ef61cb744'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:18.880236 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2275e341a8f94b0e9189832ef61cb744'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:18.888010 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2275e341a8f94b0e9189832ef61cb744'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:18.894349 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1415/2837] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:57:18 2022] DELETE /identity/v3/users/2275e341a8f94b0e9189832ef61cb744 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:57:18.898954 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8718c-7783-4085-b9e3-7b0b940c1404 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:18.899466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8718c-7783-4085-b9e3-7b0b940c1404 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:18.899466 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8718c-7783-4085-b9e3-7b0b940c1404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:18.921587 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-27b8718c-7783-4085-b9e3-7b0b940c1404 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:57:18.925014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-27b8718c-7783-4085-b9e3-7b0b940c1404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:18.983127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1423/2838] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:57:18.993709 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:18.995195 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:19.059261 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936d953-abc8-4dbe-8e86-9a0affbd6577 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:19.062858 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03830ef0-08c2-4b2e-96f3-0376f7827e47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:19.063461 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03830ef0-08c2-4b2e-96f3-0376f7827e47 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:19.063732 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03830ef0-08c2-4b2e-96f3-0376f7827e47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:19.063998 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03830ef0-08c2-4b2e-96f3-0376f7827e47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:19.073607 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03830ef0-08c2-4b2e-96f3-0376f7827e47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:19.075064 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03830ef0-08c2-4b2e-96f3-0376f7827e47 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:19.085567 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1416/2839] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:57:19.516020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8718c-7783-4085-b9e3-7b0b940c1404 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:19.517425 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-27b8718c-7783-4085-b9e3-7b0b940c1404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:19.530201 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8718c-7783-4085-b9e3-7b0b940c1404 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:19.535085 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:19.535998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:19.536253 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:19.536541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] PATH_INFO: `/v3/projects/9cb7ffe0a3b842d4947ff92e84d082cd` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:19.537828 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb7ffe0a3b842d4947ff92e84d082cd)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:19.539788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:19.676670 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb7ffe0a3b842d4947ff92e84d082cd', 'request_id': 'req-4273b8cb-e7f8-43de-9845-4d39c9e3f145'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:19.782382 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb7ffe0a3b842d4947ff92e84d082cd', 'request_id': 'req-4273b8cb-e7f8-43de-9845-4d39c9e3f145'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:19.788788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb7ffe0a3b842d4947ff92e84d082cd', 'request_id': 'req-4273b8cb-e7f8-43de-9845-4d39c9e3f145'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:19.800795 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] The token cache is being invalidate because project 9cb7ffe0a3b842d4947ff92e84d082cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:57:19.801082 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:19.802396 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:57:19.804236 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1424/2840] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:57:19 2022] DELETE /identity/v3/projects/9cb7ffe0a3b842d4947ff92e84d082cd => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:57:22.914934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b9bc4-265b-4dc0-bc72-7ad0a089ea76 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:22.921246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b9bc4-265b-4dc0-bc72-7ad0a089ea76 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:22.921246 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b9bc4-265b-4dc0-bc72-7ad0a089ea76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:22.935699 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b15b9bc4-265b-4dc0-bc72-7ad0a089ea76 None None] MFA Rules not processed for user `c1c1de0489104713a90c59b89a284321`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:57:22.936909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b15b9bc4-265b-4dc0-bc72-7ad0a089ea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:23.001268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:57:23.001268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1417/2841] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:57:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:57:23.314999 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:23.316446 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:23.372529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:57:23.378640 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273b8cb-e7f8-43de-9845-4d39c9e3f145 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:23.382635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631615f2-ed1c-4c55-92cd-d100a64d15b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:23.384910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631615f2-ed1c-4c55-92cd-d100a64d15b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:23.384910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631615f2-ed1c-4c55-92cd-d100a64d15b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:23.384910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631615f2-ed1c-4c55-92cd-d100a64d15b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:23.385622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-631615f2-ed1c-4c55-92cd-d100a64d15b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:23.396399 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631615f2-ed1c-4c55-92cd-d100a64d15b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:23.397582 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631615f2-ed1c-4c55-92cd-d100a64d15b3 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:23.407891 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1425/2842] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:57:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:57:23.862959 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b9bc4-265b-4dc0-bc72-7ad0a089ea76 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:23.864521 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b15b9bc4-265b-4dc0-bc72-7ad0a089ea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:23.944325 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b9bc4-265b-4dc0-bc72-7ad0a089ea76 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:23.949320 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997bd9d-ef87-4b4d-8cc8-bd02a3685108 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d03db170fb6460794e32669456bcdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:23.949918 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997bd9d-ef87-4b4d-8cc8-bd02a3685108 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:23.950223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997bd9d-ef87-4b4d-8cc8-bd02a3685108 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:23.950494 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997bd9d-ef87-4b4d-8cc8-bd02a3685108 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:23.952301 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0997bd9d-ef87-4b4d-8cc8-bd02a3685108 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:23.964339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997bd9d-ef87-4b4d-8cc8-bd02a3685108 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:23.965898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997bd9d-ef87-4b4d-8cc8-bd02a3685108 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:23.975907 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1418/2843] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:57:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:57:27.464559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631615f2-ed1c-4c55-92cd-d100a64d15b3 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:27.474658 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631615f2-ed1c-4c55-92cd-d100a64d15b3 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:27.478705 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e7d48-5764-469a-a044-17318e0bb133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:27.479325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7d48-5764-469a-a044-17318e0bb133 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:27.479601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7d48-5764-469a-a044-17318e0bb133 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:27.479874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7d48-5764-469a-a044-17318e0bb133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:27.481813 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3d0e7d48-5764-469a-a044-17318e0bb133 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:27.499909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e7d48-5764-469a-a044-17318e0bb133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:27.501713 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e7d48-5764-469a-a044-17318e0bb133 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:27.512135 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1426/2844] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:57:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:57:28.313775 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997bd9d-ef87-4b4d-8cc8-bd02a3685108 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:57:28.315345 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0997bd9d-ef87-4b4d-8cc8-bd02a3685108 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:28.407168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997bd9d-ef87-4b4d-8cc8-bd02a3685108 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:57:28.416917 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449b1d2-366d-42da-9d0e-9f020a29a8a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:57:28.417688 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449b1d2-366d-42da-9d0e-9f020a29a8a2 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:57:28.417966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449b1d2-366d-42da-9d0e-9f020a29a8a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:57:28.418317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449b1d2-366d-42da-9d0e-9f020a29a8a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:57:28.421113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5449b1d2-366d-42da-9d0e-9f020a29a8a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:57:28.437448 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449b1d2-366d-42da-9d0e-9f020a29a8a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:57:28.439418 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449b1d2-366d-42da-9d0e-9f020a29a8a2 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:57:28.454521 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1419/2845] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:57:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:57:37.508095 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=11 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:57:38.447593 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=73 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:58:28.435836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e7d48-5764-469a-a044-17318e0bb133 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:28.437317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3d0e7d48-5764-469a-a044-17318e0bb133 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:28.454398 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e7d48-5764-469a-a044-17318e0bb133 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:28.460046 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bd2cd-52c9-44d5-86d4-78f9ee8c077a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:28.460656 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bd2cd-52c9-44d5-86d4-78f9ee8c077a service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:28.460933 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bd2cd-52c9-44d5-86d4-78f9ee8c077a service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:28.461291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bd2cd-52c9-44d5-86d4-78f9ee8c077a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:28.463636 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bb0bd2cd-52c9-44d5-86d4-78f9ee8c077a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:28.477642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bd2cd-52c9-44d5-86d4-78f9ee8c077a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:28.479386 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bd2cd-52c9-44d5-86d4-78f9ee8c077a service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:28.489613 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1427/2846] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:58:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:58:35.870466 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449b1d2-366d-42da-9d0e-9f020a29a8a2 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:35.880777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449b1d2-366d-42da-9d0e-9f020a29a8a2 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:35.885074 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15035ee-0772-44b3-82a9-fb1b57d605c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:35.885666 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15035ee-0772-44b3-82a9-fb1b57d605c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:35.885940 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15035ee-0772-44b3-82a9-fb1b57d605c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:35.886230 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15035ee-0772-44b3-82a9-fb1b57d605c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:35.887925 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a15035ee-0772-44b3-82a9-fb1b57d605c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:35.952593 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15035ee-0772-44b3-82a9-fb1b57d605c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:35.954076 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15035ee-0772-44b3-82a9-fb1b57d605c6 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:35.964266 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1420/2847] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:58:38.484830 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:58:39.634488 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bd2cd-52c9-44d5-86d4-78f9ee8c077a service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:39.635898 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bb0bd2cd-52c9-44d5-86d4-78f9ee8c077a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:39.650862 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bd2cd-52c9-44d5-86d4-78f9ee8c077a service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:39.656003 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbeead8-96ac-4bc1-bea3-2b49e34ecbd1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d03db170fb6460794e32669456bcdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:39.656597 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbeead8-96ac-4bc1-bea3-2b49e34ecbd1 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:39.656874 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbeead8-96ac-4bc1-bea3-2b49e34ecbd1 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:39.657143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbeead8-96ac-4bc1-bea3-2b49e34ecbd1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:39.658950 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2fbeead8-96ac-4bc1-bea3-2b49e34ecbd1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:39.673601 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbeead8-96ac-4bc1-bea3-2b49e34ecbd1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:39.675316 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbeead8-96ac-4bc1-bea3-2b49e34ecbd1 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:39.685826 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1428/2848] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:58:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:58:39.760898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15035ee-0772-44b3-82a9-fb1b57d605c6 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:39.769454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a15035ee-0772-44b3-82a9-fb1b57d605c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:39.785548 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15035ee-0772-44b3-82a9-fb1b57d605c6 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:39.787083 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5629b-0591-44b6-a182-5a66b51a949a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:39.787695 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5629b-0591-44b6-a182-5a66b51a949a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:39.787968 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5629b-0591-44b6-a182-5a66b51a949a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:39.788233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5629b-0591-44b6-a182-5a66b51a949a admin admin] PATH_INFO: `/v3/users/c1c1de0489104713a90c59b89a284321` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:39.789326 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5629b-0591-44b6-a182-5a66b51a949a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c1de0489104713a90c59b89a284321)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:39.791545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5629b-0591-44b6-a182-5a66b51a949a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:40.017774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-84a5629b-0591-44b6-a182-5a66b51a949a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c1de0489104713a90c59b89a284321'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:40.030871 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-84a5629b-0591-44b6-a182-5a66b51a949a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c1de0489104713a90c59b89a284321'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:40.100937 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-84a5629b-0591-44b6-a182-5a66b51a949a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c1de0489104713a90c59b89a284321'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:40.101554 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-84a5629b-0591-44b6-a182-5a66b51a949a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c1de0489104713a90c59b89a284321'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:40.109684 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-84a5629b-0591-44b6-a182-5a66b51a949a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c1de0489104713a90c59b89a284321'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:40.115880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1421/2849] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:58:39 2022] DELETE /identity/v3/users/c1c1de0489104713a90c59b89a284321 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:58:40.446277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbeead8-96ac-4bc1-bea3-2b49e34ecbd1 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:40.459853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbeead8-96ac-4bc1-bea3-2b49e34ecbd1 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:40.530050 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:40.531016 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:40.531334 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:40.531726 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55 admin admin] PATH_INFO: `/v3/projects/c0401850bdf44aaaa6c37ed752708f53` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:40.532998 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0401850bdf44aaaa6c37ed752708f53)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:40.535067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:40.668984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0401850bdf44aaaa6c37ed752708f53', 'request_id': 'req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:40.740735 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0401850bdf44aaaa6c37ed752708f53', 'request_id': 'req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:40.747504 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0401850bdf44aaaa6c37ed752708f53', 'request_id': 'req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:40.759921 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55 admin admin] The token cache is being invalidate because project c0401850bdf44aaaa6c37ed752708f53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:58:40.762407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:40.762407 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f7afbc9-c84a-4a5d-a9fb-383d88de6e55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:40.762735 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1429/2850] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:58:40 2022] DELETE /identity/v3/projects/c0401850bdf44aaaa6c37ed752708f53 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:58:40.768651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b07b92-ba2a-42ec-894c-e8128aa19f0e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:40.769179 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b07b92-ba2a-42ec-894c-e8128aa19f0e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:40.769427 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b07b92-ba2a-42ec-894c-e8128aa19f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:40.791482 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b0b07b92-ba2a-42ec-894c-e8128aa19f0e None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:58:40.792704 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b0b07b92-ba2a-42ec-894c-e8128aa19f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:40.854690 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1422/2851] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:58:40.863984 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0007723-02b8-4a94-9801-223c1f60fc18 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:40.864268 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0007723-02b8-4a94-9801-223c1f60fc18 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:40.864544 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0007723-02b8-4a94-9801-223c1f60fc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:40.895273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b0007723-02b8-4a94-9801-223c1f60fc18 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:58:40.896885 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b0007723-02b8-4a94-9801-223c1f60fc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:40.905612 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1430/2852] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:58:40.910237 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b07b92-ba2a-42ec-894c-e8128aa19f0e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:40.920862 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b07b92-ba2a-42ec-894c-e8128aa19f0e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:40.925577 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19aaeab-598d-460e-9544-377eb8610f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:40.926205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19aaeab-598d-460e-9544-377eb8610f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:40.926510 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19aaeab-598d-460e-9544-377eb8610f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:40.926779 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19aaeab-598d-460e-9544-377eb8610f28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:40.927432 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19aaeab-598d-460e-9544-377eb8610f28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:58:40.927731 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19aaeab-598d-460e-9544-377eb8610f28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:40.929331 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19aaeab-598d-460e-9544-377eb8610f28 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:40.931525 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:58:40.943268 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1423/2853] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:58:40.947655 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0007723-02b8-4a94-9801-223c1f60fc18 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:40.957044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0007723-02b8-4a94-9801-223c1f60fc18 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:40.962533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5c666-79c4-4c93-8bad-8a03ee7e6366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:40.963156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5c666-79c4-4c93-8bad-8a03ee7e6366 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:40.963427 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5c666-79c4-4c93-8bad-8a03ee7e6366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:40.963693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5c666-79c4-4c93-8bad-8a03ee7e6366 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:40.964343 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5c666-79c4-4c93-8bad-8a03ee7e6366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:40.966258 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5c666-79c4-4c93-8bad-8a03ee7e6366 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:41.061932 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1431/2854] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:58:40 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:58:41.066092 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19aaeab-598d-460e-9544-377eb8610f28 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:41.076573 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19aaeab-598d-460e-9544-377eb8610f28 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:41.133287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d3b82-6ba3-4f79-a51c-bb167e86ea75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:41.134428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d3b82-6ba3-4f79-a51c-bb167e86ea75 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:41.134705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d3b82-6ba3-4f79-a51c-bb167e86ea75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:41.134966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d3b82-6ba3-4f79-a51c-bb167e86ea75 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:41.135583 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d3b82-6ba3-4f79-a51c-bb167e86ea75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:41.137467 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d3b82-6ba3-4f79-a51c-bb167e86ea75 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:41.209768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1424/2855] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:58:41 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:58:41.214261 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5c666-79c4-4c93-8bad-8a03ee7e6366 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:41.224741 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5c666-79c4-4c93-8bad-8a03ee7e6366 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:41.229923 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977a86e-467d-49f5-86e7-150e5ba1e94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:41.230550 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977a86e-467d-49f5-86e7-150e5ba1e94e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:41.230868 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977a86e-467d-49f5-86e7-150e5ba1e94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:41.231140 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977a86e-467d-49f5-86e7-150e5ba1e94e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:41.231801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977a86e-467d-49f5-86e7-150e5ba1e94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:41.233370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977a86e-467d-49f5-86e7-150e5ba1e94e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:41.244325 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1432/2856] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:58:41.248447 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d3b82-6ba3-4f79-a51c-bb167e86ea75 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:41.258882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d3b82-6ba3-4f79-a51c-bb167e86ea75 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:41.264223 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9427c3-e3fd-40b2-98f7-eb6a23a7328b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:41.264843 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9427c3-e3fd-40b2-98f7-eb6a23a7328b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:41.265122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9427c3-e3fd-40b2-98f7-eb6a23a7328b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:41.265449 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9427c3-e3fd-40b2-98f7-eb6a23a7328b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:41.266029 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9427c3-e3fd-40b2-98f7-eb6a23a7328b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:41.267677 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9427c3-e3fd-40b2-98f7-eb6a23a7328b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:41.278966 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1425/2857] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:58:41.283411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977a86e-467d-49f5-86e7-150e5ba1e94e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:41.294099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977a86e-467d-49f5-86e7-150e5ba1e94e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:41.298673 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84809156-e480-4213-8418-b128e9bd9b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:41.299310 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84809156-e480-4213-8418-b128e9bd9b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:41.299621 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84809156-e480-4213-8418-b128e9bd9b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:41.299892 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84809156-e480-4213-8418-b128e9bd9b14 admin admin] PATH_INFO: `/v3/projects/d21bd23c547f40fb816261885a0f642c/users/d62895326fbd44448a27b5981b77b126/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:41.316196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84809156-e480-4213-8418-b128e9bd9b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d21bd23c547f40fb816261885a0f642c, user_id=d62895326fbd44448a27b5981b77b126, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:41.318857 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84809156-e480-4213-8418-b128e9bd9b14 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:41.438533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1433/2858] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:58:41 2022] PUT /identity/v3/projects/d21bd23c547f40fb816261885a0f642c/users/d62895326fbd44448a27b5981b77b126/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:58:41.448866 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5a7df-3ff1-4df2-8696-7280ecc5df51 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:41.449507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5a7df-3ff1-4df2-8696-7280ecc5df51 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:41.449507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5a7df-3ff1-4df2-8696-7280ecc5df51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:41.477188 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2ca5a7df-3ff1-4df2-8696-7280ecc5df51 None None] MFA Rules not processed for user `d62895326fbd44448a27b5981b77b126`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:58:41.479066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2ca5a7df-3ff1-4df2-8696-7280ecc5df51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:41.541856 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1426/2859] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:58:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:58:41.553085 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84809156-e480-4213-8418-b128e9bd9b14 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:41.554763 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-84809156-e480-4213-8418-b128e9bd9b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:41.569515 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:58:41.631327 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84809156-e480-4213-8418-b128e9bd9b14 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:41.638484 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d55002-6eec-436c-a8a5-d520d979ab25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:41.639127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d55002-6eec-436c-a8a5-d520d979ab25 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:41.639392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d55002-6eec-436c-a8a5-d520d979ab25 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:41.639694 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d55002-6eec-436c-a8a5-d520d979ab25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:41.647940 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d55002-6eec-436c-a8a5-d520d979ab25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:41.649532 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d55002-6eec-436c-a8a5-d520d979ab25 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:41.659047 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1434/2860] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:58:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:58:49.942659 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5a7df-3ff1-4df2-8696-7280ecc5df51 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:50.004849 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5a7df-3ff1-4df2-8696-7280ecc5df51 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:50.009809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:50.010463 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:50.010737 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:50.011003 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] PATH_INFO: `/v3/users/d62895326fbd44448a27b5981b77b126` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:50.012119 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62895326fbd44448a27b5981b77b126)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:50.014029 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:50.016566 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:58:50.188909 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62895326fbd44448a27b5981b77b126'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:50.201748 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62895326fbd44448a27b5981b77b126'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:50.260659 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62895326fbd44448a27b5981b77b126'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:50.261215 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62895326fbd44448a27b5981b77b126'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:50.269135 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62895326fbd44448a27b5981b77b126'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:50.275610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1427/2861] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:58:49 2022] DELETE /identity/v3/users/d62895326fbd44448a27b5981b77b126 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:58:50.280169 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04867703-c0e9-4528-b40d-b2c1bd6fd37d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:50.280719 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04867703-c0e9-4528-b40d-b2c1bd6fd37d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:50.280908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04867703-c0e9-4528-b40d-b2c1bd6fd37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:50.300400 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-04867703-c0e9-4528-b40d-b2c1bd6fd37d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:58:50.301605 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-04867703-c0e9-4528-b40d-b2c1bd6fd37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:50.361706 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1435/2862] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:58:50.372429 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:50.373851 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:50.434098 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f6b2c-309d-4342-a8d7-f9a915fd00ad admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:50.437983 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66defe5c-5116-46e6-b482-3f02302702b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:50.438622 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66defe5c-5116-46e6-b482-3f02302702b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:50.438895 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66defe5c-5116-46e6-b482-3f02302702b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:50.439161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66defe5c-5116-46e6-b482-3f02302702b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:50.449209 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66defe5c-5116-46e6-b482-3f02302702b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:50.450705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66defe5c-5116-46e6-b482-3f02302702b1 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:50.460521 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1428/2863] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:58:50.863142 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04867703-c0e9-4528-b40d-b2c1bd6fd37d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:50.864690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-04867703-c0e9-4528-b40d-b2c1bd6fd37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:50.879639 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04867703-c0e9-4528-b40d-b2c1bd6fd37d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:50.884553 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e126877-8c63-41bb-8b0a-991e88070453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:50.885194 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e126877-8c63-41bb-8b0a-991e88070453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:50.885480 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e126877-8c63-41bb-8b0a-991e88070453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:50.885737 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e126877-8c63-41bb-8b0a-991e88070453 admin admin] PATH_INFO: `/v3/projects/d21bd23c547f40fb816261885a0f642c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:50.886881 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e126877-8c63-41bb-8b0a-991e88070453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d21bd23c547f40fb816261885a0f642c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:50.888935 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e126877-8c63-41bb-8b0a-991e88070453 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:51.027219 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e126877-8c63-41bb-8b0a-991e88070453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd21bd23c547f40fb816261885a0f642c', 'request_id': 'req-3e126877-8c63-41bb-8b0a-991e88070453'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:51.100263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e126877-8c63-41bb-8b0a-991e88070453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd21bd23c547f40fb816261885a0f642c', 'request_id': 'req-3e126877-8c63-41bb-8b0a-991e88070453'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:51.107048 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e126877-8c63-41bb-8b0a-991e88070453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd21bd23c547f40fb816261885a0f642c', 'request_id': 'req-3e126877-8c63-41bb-8b0a-991e88070453'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:51.119485 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e126877-8c63-41bb-8b0a-991e88070453 admin admin] The token cache is being invalidate because project d21bd23c547f40fb816261885a0f642c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:58:51.119757 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e126877-8c63-41bb-8b0a-991e88070453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:51.120462 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e126877-8c63-41bb-8b0a-991e88070453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:58:51.121945 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1436/2864] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:58:50 2022] DELETE /identity/v3/projects/d21bd23c547f40fb816261885a0f642c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:58:51.128498 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad3847-854b-47d0-ae85-66eb5053dbcd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:51.129019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad3847-854b-47d0-ae85-66eb5053dbcd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:51.129019 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad3847-854b-47d0-ae85-66eb5053dbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:51.150445 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a7ad3847-854b-47d0-ae85-66eb5053dbcd None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:58:51.154476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a7ad3847-854b-47d0-ae85-66eb5053dbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:51.213304 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1429/2865] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:58:51.221294 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597d293-cf83-450b-939e-0c987964f149 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:51.221576 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597d293-cf83-450b-939e-0c987964f149 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:51.221853 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597d293-cf83-450b-939e-0c987964f149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:51.242736 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c597d293-cf83-450b-939e-0c987964f149 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:58:51.243960 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c597d293-cf83-450b-939e-0c987964f149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:51.251910 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1437/2866] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:58:51.256413 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad3847-854b-47d0-ae85-66eb5053dbcd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:51.271626 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad3847-854b-47d0-ae85-66eb5053dbcd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:51.276956 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a0991-d3b2-44eb-a401-17527acbda73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:51.277627 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a0991-d3b2-44eb-a401-17527acbda73 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:51.277930 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a0991-d3b2-44eb-a401-17527acbda73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:51.278239 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a0991-d3b2-44eb-a401-17527acbda73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:51.278969 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a0991-d3b2-44eb-a401-17527acbda73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:58:51.279274 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a0991-d3b2-44eb-a401-17527acbda73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:51.280867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a0991-d3b2-44eb-a401-17527acbda73 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:51.293489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1430/2867] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:58:51.298003 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597d293-cf83-450b-939e-0c987964f149 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:51.307538 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597d293-cf83-450b-939e-0c987964f149 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:51.312230 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee3481-6210-43f7-b568-87fcd73cdc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:51.312824 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee3481-6210-43f7-b568-87fcd73cdc8f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:51.313121 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee3481-6210-43f7-b568-87fcd73cdc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:51.313387 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee3481-6210-43f7-b568-87fcd73cdc8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:51.314017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee3481-6210-43f7-b568-87fcd73cdc8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:51.315982 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee3481-6210-43f7-b568-87fcd73cdc8f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:51.331241 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:58:51.400593 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1438/2868] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:58:51 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:58:51.404810 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a0991-d3b2-44eb-a401-17527acbda73 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:51.415881 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a0991-d3b2-44eb-a401-17527acbda73 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:51.424675 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:58:51.473915 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4851580-9f59-4ccd-a70d-1e11c4a902cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:51.474553 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851580-9f59-4ccd-a70d-1e11c4a902cc admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:51.474830 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851580-9f59-4ccd-a70d-1e11c4a902cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:51.475099 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4851580-9f59-4ccd-a70d-1e11c4a902cc admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:51.475743 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4851580-9f59-4ccd-a70d-1e11c4a902cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:51.478287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4851580-9f59-4ccd-a70d-1e11c4a902cc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:51.621597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1431/2869] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:58:51 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:58:51.627183 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee3481-6210-43f7-b568-87fcd73cdc8f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:51.639413 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee3481-6210-43f7-b568-87fcd73cdc8f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:51.644124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a8011-1dae-45a8-a85e-d640179f3b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:51.644737 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a8011-1dae-45a8-a85e-d640179f3b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:51.645014 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a8011-1dae-45a8-a85e-d640179f3b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:51.645280 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a8011-1dae-45a8-a85e-d640179f3b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:51.645916 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a8011-1dae-45a8-a85e-d640179f3b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:51.647693 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a8011-1dae-45a8-a85e-d640179f3b23 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:51.659232 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1439/2870] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:58:51.663315 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4851580-9f59-4ccd-a70d-1e11c4a902cc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:51.674280 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4851580-9f59-4ccd-a70d-1e11c4a902cc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:51.681373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88296b-3892-48c6-8d0e-d0d75b35bd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:51.681373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88296b-3892-48c6-8d0e-d0d75b35bd49 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:51.681373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88296b-3892-48c6-8d0e-d0d75b35bd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:51.681373 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88296b-3892-48c6-8d0e-d0d75b35bd49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:51.681867 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88296b-3892-48c6-8d0e-d0d75b35bd49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:51.682982 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88296b-3892-48c6-8d0e-d0d75b35bd49 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:51.701615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1432/2871] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:58:51.705942 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a8011-1dae-45a8-a85e-d640179f3b23 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:51.719642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a8011-1dae-45a8-a85e-d640179f3b23 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:51.724610 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65384c-dba9-4a6d-9ec4-c98ad5a2c1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:51.725267 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65384c-dba9-4a6d-9ec4-c98ad5a2c1d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:51.725547 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65384c-dba9-4a6d-9ec4-c98ad5a2c1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:51.725818 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65384c-dba9-4a6d-9ec4-c98ad5a2c1d2 admin admin] PATH_INFO: `/v3/projects/688d5fb5a9c0487e8374c29000c8de12/users/f0c29dcc98a54224bc17163d4e66c6d2/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:51.741879 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65384c-dba9-4a6d-9ec4-c98ad5a2c1d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=688d5fb5a9c0487e8374c29000c8de12, user_id=f0c29dcc98a54224bc17163d4e66c6d2, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:51.744615 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65384c-dba9-4a6d-9ec4-c98ad5a2c1d2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:51.827156 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1440/2872] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:58:51 2022] PUT /identity/v3/projects/688d5fb5a9c0487e8374c29000c8de12/users/f0c29dcc98a54224bc17163d4e66c6d2/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:58:51.839217 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaad745-b2cc-4896-bb87-2601a9cf980e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:51.839622 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaad745-b2cc-4896-bb87-2601a9cf980e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:51.839961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaad745-b2cc-4896-bb87-2601a9cf980e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:51.869122 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7eaad745-b2cc-4896-bb87-2601a9cf980e None None] MFA Rules not processed for user `f0c29dcc98a54224bc17163d4e66c6d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:58:51.870365 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7eaad745-b2cc-4896-bb87-2601a9cf980e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:51.936777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1433/2873] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:58:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:58:51.941267 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65384c-dba9-4a6d-9ec4-c98ad5a2c1d2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:52.011541 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65384c-dba9-4a6d-9ec4-c98ad5a2c1d2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:52.016244 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219069d-af01-4df4-a43d-153d836a623e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:52.016843 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219069d-af01-4df4-a43d-153d836a623e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:52.017115 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219069d-af01-4df4-a43d-153d836a623e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:52.017380 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219069d-af01-4df4-a43d-153d836a623e admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:52.018004 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219069d-af01-4df4-a43d-153d836a623e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:52.019927 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219069d-af01-4df4-a43d-153d836a623e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:52.120519 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1441/2874] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:58:51 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:58:52.124679 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaad745-b2cc-4896-bb87-2601a9cf980e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:52.136369 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaad745-b2cc-4896-bb87-2601a9cf980e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:52.192990 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b809e-7e51-4ae8-b993-7f61d664c8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:52.193502 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b809e-7e51-4ae8-b993-7f61d664c8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:52.193778 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b809e-7e51-4ae8-b993-7f61d664c8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:52.194047 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b809e-7e51-4ae8-b993-7f61d664c8c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:52.194705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b809e-7e51-4ae8-b993-7f61d664c8c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:52.197220 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b809e-7e51-4ae8-b993-7f61d664c8c0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:52.301545 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1434/2875] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:58:52 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:58:52.306675 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219069d-af01-4df4-a43d-153d836a623e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:52.317536 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219069d-af01-4df4-a43d-153d836a623e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:52.322222 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb0230-d840-44df-b624-05d8a598d461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:52.323955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb0230-d840-44df-b624-05d8a598d461 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:52.323955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb0230-d840-44df-b624-05d8a598d461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:52.323955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb0230-d840-44df-b624-05d8a598d461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:52.323955 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb0230-d840-44df-b624-05d8a598d461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:52.325654 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb0230-d840-44df-b624-05d8a598d461 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:52.336450 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1442/2876] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:58:52.340528 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b809e-7e51-4ae8-b993-7f61d664c8c0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:52.351070 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b809e-7e51-4ae8-b993-7f61d664c8c0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:52.356454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82d7f3-e5b4-4e46-b529-e1b352fa3b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:52.357068 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82d7f3-e5b4-4e46-b529-e1b352fa3b81 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:52.357341 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82d7f3-e5b4-4e46-b529-e1b352fa3b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:52.357610 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82d7f3-e5b4-4e46-b529-e1b352fa3b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:52.358265 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82d7f3-e5b4-4e46-b529-e1b352fa3b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:52.359900 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82d7f3-e5b4-4e46-b529-e1b352fa3b81 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:52.361992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:58:52.371213 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1435/2877] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:58:52.375429 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb0230-d840-44df-b624-05d8a598d461 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:52.386314 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb0230-d840-44df-b624-05d8a598d461 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:52.390890 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3118b66-8f35-4e92-b556-060baf828a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:52.391571 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3118b66-8f35-4e92-b556-060baf828a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:52.391805 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3118b66-8f35-4e92-b556-060baf828a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:52.392076 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3118b66-8f35-4e92-b556-060baf828a04 admin admin] PATH_INFO: `/v3/projects/82ed7bf6c8fc4c3ba164a71df27d2001/users/5f23c98dc16542549c61a2e3510dd8ba/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:52.406820 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3118b66-8f35-4e92-b556-060baf828a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ed7bf6c8fc4c3ba164a71df27d2001, user_id=5f23c98dc16542549c61a2e3510dd8ba, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:52.409508 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3118b66-8f35-4e92-b556-060baf828a04 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:52.467235 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1443/2878] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:58:52 2022] PUT /identity/v3/projects/82ed7bf6c8fc4c3ba164a71df27d2001/users/5f23c98dc16542549c61a2e3510dd8ba/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:58:52.478221 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd1408-276b-469c-ae7d-233cda75001c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:52.478504 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd1408-276b-469c-ae7d-233cda75001c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:52.478758 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd1408-276b-469c-ae7d-233cda75001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:52.509099 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cdcd1408-276b-469c-ae7d-233cda75001c None None] MFA Rules not processed for user `5f23c98dc16542549c61a2e3510dd8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:58:52.510402 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cdcd1408-276b-469c-ae7d-233cda75001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:52.578686 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1436/2879] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:58:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:58:52.583347 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3118b66-8f35-4e92-b556-060baf828a04 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:52.648266 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3118b66-8f35-4e92-b556-060baf828a04 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:52.653527 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0632c-ad8a-4368-b4b6-dad883e7610e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:52.654157 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0632c-ad8a-4368-b4b6-dad883e7610e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:52.654451 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0632c-ad8a-4368-b4b6-dad883e7610e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:52.654718 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0632c-ad8a-4368-b4b6-dad883e7610e admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:52.655354 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0632c-ad8a-4368-b4b6-dad883e7610e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:52.657270 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0632c-ad8a-4368-b4b6-dad883e7610e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:52.673140 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:58:52.737265 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1444/2880] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:58:52 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:58:52.742121 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd1408-276b-469c-ae7d-233cda75001c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:52.753841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd1408-276b-469c-ae7d-233cda75001c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:52.813181 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c249c-808c-48e2-a7ef-d2b7d5236637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:52.813799 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c249c-808c-48e2-a7ef-d2b7d5236637 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:52.814077 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c249c-808c-48e2-a7ef-d2b7d5236637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:52.814375 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c249c-808c-48e2-a7ef-d2b7d5236637 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:52.815026 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c249c-808c-48e2-a7ef-d2b7d5236637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:52.816944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c249c-808c-48e2-a7ef-d2b7d5236637 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:52.959291 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1437/2881] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:58:52 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:58:52.963822 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0632c-ad8a-4368-b4b6-dad883e7610e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:52.974196 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0632c-ad8a-4368-b4b6-dad883e7610e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:52.979216 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0343fae-22d2-4e25-8cf9-5ef9030ddfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:52.979800 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0343fae-22d2-4e25-8cf9-5ef9030ddfa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:52.980074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0343fae-22d2-4e25-8cf9-5ef9030ddfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:52.980423 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0343fae-22d2-4e25-8cf9-5ef9030ddfa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:52.980968 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0343fae-22d2-4e25-8cf9-5ef9030ddfa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:52.982586 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0343fae-22d2-4e25-8cf9-5ef9030ddfa5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:52.994446 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1445/2882] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:58:52.998537 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c249c-808c-48e2-a7ef-d2b7d5236637 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:53.009058 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c249c-808c-48e2-a7ef-d2b7d5236637 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:53.013708 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b41bc-5745-4520-9259-d2d179bed289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:53.014386 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b41bc-5745-4520-9259-d2d179bed289 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:53.014662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b41bc-5745-4520-9259-d2d179bed289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:53.014932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b41bc-5745-4520-9259-d2d179bed289 admin admin] PATH_INFO: `/v3/projects/a1deeb621bab45daaaca2511b6b769cf/users/56a6604f3f4648478921e0abe0793ee5/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:53.030279 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b41bc-5745-4520-9259-d2d179bed289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1deeb621bab45daaaca2511b6b769cf, user_id=56a6604f3f4648478921e0abe0793ee5, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:53.032916 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b41bc-5745-4520-9259-d2d179bed289 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:53.084576 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1438/2883] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:58:52 2022] PUT /identity/v3/projects/a1deeb621bab45daaaca2511b6b769cf/users/56a6604f3f4648478921e0abe0793ee5/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:58:53.089163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0343fae-22d2-4e25-8cf9-5ef9030ddfa5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:53.099525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0343fae-22d2-4e25-8cf9-5ef9030ddfa5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:53.157860 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a77c5c-4540-4531-bb48-23ed12fbac0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:53.158559 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a77c5c-4540-4531-bb48-23ed12fbac0b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:53.158881 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a77c5c-4540-4531-bb48-23ed12fbac0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:53.159191 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a77c5c-4540-4531-bb48-23ed12fbac0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:53.160017 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a77c5c-4540-4531-bb48-23ed12fbac0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:53.161807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a77c5c-4540-4531-bb48-23ed12fbac0b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:53.172224 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1446/2884] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:58:53.176624 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b41bc-5745-4520-9259-d2d179bed289 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:53.193860 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b41bc-5745-4520-9259-d2d179bed289 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:53.202006 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d41e8-14e6-4d09-89dd-6f2070f735be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:53.203148 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d41e8-14e6-4d09-89dd-6f2070f735be admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:53.203630 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d41e8-14e6-4d09-89dd-6f2070f735be admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:53.204113 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969d41e8-14e6-4d09-89dd-6f2070f735be admin admin] PATH_INFO: `/v3/projects/a1deeb621bab45daaaca2511b6b769cf/users/56a6604f3f4648478921e0abe0793ee5/roles/020bc02dca71485bab2312442adaa179` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:53.207503 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d41e8-14e6-4d09-89dd-6f2070f735be admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1deeb621bab45daaaca2511b6b769cf, user_id=56a6604f3f4648478921e0abe0793ee5, role_id=020bc02dca71485bab2312442adaa179)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:53.213535 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969d41e8-14e6-4d09-89dd-6f2070f735be admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:53.239026 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1439/2885] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:58:53 2022] PUT /identity/v3/projects/a1deeb621bab45daaaca2511b6b769cf/users/56a6604f3f4648478921e0abe0793ee5/roles/020bc02dca71485bab2312442adaa179 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:58:53.251819 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8e6ba-0d4a-485a-83eb-840344ca73be None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:53.252042 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8e6ba-0d4a-485a-83eb-840344ca73be None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:53.252275 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8e6ba-0d4a-485a-83eb-840344ca73be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:53.289053 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-70b8e6ba-0d4a-485a-83eb-840344ca73be None None] MFA Rules not processed for user `56a6604f3f4648478921e0abe0793ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:58:53.291353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-70b8e6ba-0d4a-485a-83eb-840344ca73be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:53.370836 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1447/2886] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:58:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:58:53.383992 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d41e8-14e6-4d09-89dd-6f2070f735be admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:53.385794 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-969d41e8-14e6-4d09-89dd-6f2070f735be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:53.434662 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:58:53.449942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969d41e8-14e6-4d09-89dd-6f2070f735be admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:53.453878 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ee053-7045-4712-8c23-34d4b396b5d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:53.454531 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ee053-7045-4712-8c23-34d4b396b5d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:53.454812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ee053-7045-4712-8c23-34d4b396b5d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:53.455085 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ee053-7045-4712-8c23-34d4b396b5d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:53.513431 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ee053-7045-4712-8c23-34d4b396b5d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:53.514814 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ee053-7045-4712-8c23-34d4b396b5d8 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:53.524319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1440/2887] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:58:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:58:55.639215 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8e6ba-0d4a-485a-83eb-840344ca73be None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:55.640622 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-70b8e6ba-0d4a-485a-83eb-840344ca73be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:58:55.653575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8e6ba-0d4a-485a-83eb-840344ca73be None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:55.657182 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447b35c-7cb4-47bf-b62e-8f56304eac12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:55.657784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447b35c-7cb4-47bf-b62e-8f56304eac12 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:55.658053 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447b35c-7cb4-47bf-b62e-8f56304eac12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:55.658354 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447b35c-7cb4-47bf-b62e-8f56304eac12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:55.720732 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447b35c-7cb4-47bf-b62e-8f56304eac12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:55.722206 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447b35c-7cb4-47bf-b62e-8f56304eac12 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:55.732257 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1448/2888] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:58:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:58:57.199335 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ee053-7045-4712-8c23-34d4b396b5d8 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:58:57.208457 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ee053-7045-4712-8c23-34d4b396b5d8 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:58:57.212717 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228782f5-7896-47a5-8d7c-584f0f344075 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:58:57.213403 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228782f5-7896-47a5-8d7c-584f0f344075 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:58:57.213701 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228782f5-7896-47a5-8d7c-584f0f344075 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:58:57.213967 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228782f5-7896-47a5-8d7c-584f0f344075 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:58:57.224013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228782f5-7896-47a5-8d7c-584f0f344075 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:58:57.225550 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228782f5-7896-47a5-8d7c-584f0f344075 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:58:57.236822 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1441/2889] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:58:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:59:02.033253 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447b35c-7cb4-47bf-b62e-8f56304eac12 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:02.037591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:02.108348 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447b35c-7cb4-47bf-b62e-8f56304eac12 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:02.113533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:02.114522 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:02.114877 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:02.115166 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] PATH_INFO: `/v3/users/f0c29dcc98a54224bc17163d4e66c6d2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:02.116648 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0c29dcc98a54224bc17163d4e66c6d2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:02.122798 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:02.352184 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0c29dcc98a54224bc17163d4e66c6d2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:02.353302 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0c29dcc98a54224bc17163d4e66c6d2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:02.449033 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0c29dcc98a54224bc17163d4e66c6d2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:02.458695 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0c29dcc98a54224bc17163d4e66c6d2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:02.478020 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0c29dcc98a54224bc17163d4e66c6d2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:02.485015 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1449/2890] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:59:02 2022] DELETE /identity/v3/users/f0c29dcc98a54224bc17163d4e66c6d2 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:59:02.489887 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b17745-8d68-4449-80f7-0cd66effb522 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:02.490393 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b17745-8d68-4449-80f7-0cd66effb522 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:02.490511 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b17745-8d68-4449-80f7-0cd66effb522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:02.513802 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-38b17745-8d68-4449-80f7-0cd66effb522 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:59:02.515464 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-38b17745-8d68-4449-80f7-0cd66effb522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:02.585696 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1442/2891] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:59:02.596284 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:02.601412 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:02.668698 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2c480-dcb1-43f5-b1b0-261c71503f60 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:02.672603 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f412b34-e41a-4464-a34f-7175722808c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:02.673292 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f412b34-e41a-4464-a34f-7175722808c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:02.673488 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f412b34-e41a-4464-a34f-7175722808c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:02.673876 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f412b34-e41a-4464-a34f-7175722808c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:02.684277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f412b34-e41a-4464-a34f-7175722808c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:02.687347 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f412b34-e41a-4464-a34f-7175722808c1 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:02.696167 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1450/2892] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:59:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:59:03.157719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b17745-8d68-4449-80f7-0cd66effb522 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:03.157719 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-38b17745-8d68-4449-80f7-0cd66effb522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:03.169034 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b17745-8d68-4449-80f7-0cd66effb522 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:03.174205 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d661ea-28aa-4391-9bf3-01d90eb2f841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:03.174857 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d661ea-28aa-4391-9bf3-01d90eb2f841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:03.175063 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d661ea-28aa-4391-9bf3-01d90eb2f841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:03.175334 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d661ea-28aa-4391-9bf3-01d90eb2f841 admin admin] PATH_INFO: `/v3/projects/688d5fb5a9c0487e8374c29000c8de12` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:03.176476 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d661ea-28aa-4391-9bf3-01d90eb2f841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=688d5fb5a9c0487e8374c29000c8de12)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:03.178415 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d661ea-28aa-4391-9bf3-01d90eb2f841 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:03.337768 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-98d661ea-28aa-4391-9bf3-01d90eb2f841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '688d5fb5a9c0487e8374c29000c8de12', 'request_id': 'req-98d661ea-28aa-4391-9bf3-01d90eb2f841'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:03.403344 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-98d661ea-28aa-4391-9bf3-01d90eb2f841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '688d5fb5a9c0487e8374c29000c8de12', 'request_id': 'req-98d661ea-28aa-4391-9bf3-01d90eb2f841'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:03.409721 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-98d661ea-28aa-4391-9bf3-01d90eb2f841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '688d5fb5a9c0487e8374c29000c8de12', 'request_id': 'req-98d661ea-28aa-4391-9bf3-01d90eb2f841'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:03.422552 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-98d661ea-28aa-4391-9bf3-01d90eb2f841 admin admin] The token cache is being invalidate because project 688d5fb5a9c0487e8374c29000c8de12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:59:03.422844 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-98d661ea-28aa-4391-9bf3-01d90eb2f841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:03.423507 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-98d661ea-28aa-4391-9bf3-01d90eb2f841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:03.425086 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1443/2893] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:59:03 2022] DELETE /identity/v3/projects/688d5fb5a9c0487e8374c29000c8de12 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:59:03.429127 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f412b34-e41a-4464-a34f-7175722808c1 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:03.439433 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f412b34-e41a-4464-a34f-7175722808c1 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:03.463741 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:03.500659 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:03.501262 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:03.501533 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:03.501798 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] PATH_INFO: `/v3/users/5f23c98dc16542549c61a2e3510dd8ba` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:03.502937 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f23c98dc16542549c61a2e3510dd8ba)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:03.504847 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:03.576198 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd097d2-7180-4dc2-9b30-40fd1490ed76 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:03.578757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd097d2-7180-4dc2-9b30-40fd1490ed76 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:03.578757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd097d2-7180-4dc2-9b30-40fd1490ed76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:03.585023 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:03.598903 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1fd097d2-7180-4dc2-9b30-40fd1490ed76 None None] MFA Rules not processed for user `ed50f5d8261d44b487390de092ef9136`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:59:03.600153 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1fd097d2-7180-4dc2-9b30-40fd1490ed76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:03.658720 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1444/2894] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:59:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:59:03.711567 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f23c98dc16542549c61a2e3510dd8ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:03.713273 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f23c98dc16542549c61a2e3510dd8ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:03.797901 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f23c98dc16542549c61a2e3510dd8ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:03.807975 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f23c98dc16542549c61a2e3510dd8ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:03.820477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f23c98dc16542549c61a2e3510dd8ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:03.827478 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1451/2895] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:59:03 2022] DELETE /identity/v3/users/5f23c98dc16542549c61a2e3510dd8ba => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:59:04.284899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd097d2-7180-4dc2-9b30-40fd1490ed76 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:04.284899 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1fd097d2-7180-4dc2-9b30-40fd1490ed76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:04.299158 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd097d2-7180-4dc2-9b30-40fd1490ed76 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:04.370588 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:04.371236 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:04.371513 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:04.371782 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] PATH_INFO: `/v3/projects/82ed7bf6c8fc4c3ba164a71df27d2001` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:04.372993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ed7bf6c8fc4c3ba164a71df27d2001)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:04.375029 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:04.481399 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ed7bf6c8fc4c3ba164a71df27d2001', 'request_id': 'req-f517a4d3-dc7b-42ac-8f63-783098a125d6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:04.547278 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ed7bf6c8fc4c3ba164a71df27d2001', 'request_id': 'req-f517a4d3-dc7b-42ac-8f63-783098a125d6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:04.554957 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ed7bf6c8fc4c3ba164a71df27d2001', 'request_id': 'req-f517a4d3-dc7b-42ac-8f63-783098a125d6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:04.570184 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] The token cache is being invalidate because project 82ed7bf6c8fc4c3ba164a71df27d2001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:59:04.570528 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:04.571206 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:04.572757 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1445/2896] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:59:04 2022] DELETE /identity/v3/projects/82ed7bf6c8fc4c3ba164a71df27d2001 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:59:04.578690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:04.581207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:04.600899 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e269e-3eea-4b43-b202-9757a5b5509a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:04.674645 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261ba84-04de-4317-ae40-6b1fe18e637c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:04.675291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261ba84-04de-4317-ae40-6b1fe18e637c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:04.675564 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261ba84-04de-4317-ae40-6b1fe18e637c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:04.675832 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261ba84-04de-4317-ae40-6b1fe18e637c admin admin] PATH_INFO: `/v3/users/56a6604f3f4648478921e0abe0793ee5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:04.677124 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261ba84-04de-4317-ae40-6b1fe18e637c admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a6604f3f4648478921e0abe0793ee5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:04.679591 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261ba84-04de-4317-ae40-6b1fe18e637c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:04.910098 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:04.913167 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2261ba84-04de-4317-ae40-6b1fe18e637c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a6604f3f4648478921e0abe0793ee5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:04.913832 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2261ba84-04de-4317-ae40-6b1fe18e637c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a6604f3f4648478921e0abe0793ee5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:05.014704 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2261ba84-04de-4317-ae40-6b1fe18e637c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a6604f3f4648478921e0abe0793ee5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:05.026063 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2261ba84-04de-4317-ae40-6b1fe18e637c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a6604f3f4648478921e0abe0793ee5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:05.040353 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2261ba84-04de-4317-ae40-6b1fe18e637c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a6604f3f4648478921e0abe0793ee5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:05.047044 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1452/2897] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:59:04 2022] DELETE /identity/v3/users/56a6604f3f4648478921e0abe0793ee5 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:59:05.650714 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:05.652597 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:05.666935 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517a4d3-dc7b-42ac-8f63-783098a125d6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:05.723286 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:05.742977 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:05.744204 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:05.744481 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:05.744750 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55 admin admin] PATH_INFO: `/v3/projects/a1deeb621bab45daaaca2511b6b769cf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:05.745880 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1deeb621bab45daaaca2511b6b769cf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:05.747821 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:05.887809 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1deeb621bab45daaaca2511b6b769cf', 'request_id': 'req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:05.986755 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1deeb621bab45daaaca2511b6b769cf', 'request_id': 'req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:05.993084 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1deeb621bab45daaaca2511b6b769cf', 'request_id': 'req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:06.005443 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55 admin admin] The token cache is being invalidate because project a1deeb621bab45daaaca2511b6b769cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:59:06.005753 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:06.006435 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8d3550bf-6adb-4b1a-9847-0cb2a000dc55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:06.007991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1446/2898] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:59:05 2022] DELETE /identity/v3/projects/a1deeb621bab45daaaca2511b6b769cf => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:59:06.013164 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03ea43-950c-48d4-aecc-98ef9d6072f1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:06.013697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03ea43-950c-48d4-aecc-98ef9d6072f1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:06.013697 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03ea43-950c-48d4-aecc-98ef9d6072f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:06.034952 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5f03ea43-950c-48d4-aecc-98ef9d6072f1 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:59:06.036245 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5f03ea43-950c-48d4-aecc-98ef9d6072f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:06.099618 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1453/2899] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:59:06.165941 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdf33d-1cdc-4c7d-ae5e-c978a052a5ba None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:06.166371 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdf33d-1cdc-4c7d-ae5e-c978a052a5ba None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:06.166651 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdf33d-1cdc-4c7d-ae5e-c978a052a5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:06.187072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-46bdf33d-1cdc-4c7d-ae5e-c978a052a5ba None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:59:06.188290 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-46bdf33d-1cdc-4c7d-ae5e-c978a052a5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:06.196400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1447/2900] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:59:06.202411 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03ea43-950c-48d4-aecc-98ef9d6072f1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:06.212752 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03ea43-950c-48d4-aecc-98ef9d6072f1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:06.217444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb9b32-267f-4657-81cd-ab4996d4a813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:06.218097 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb9b32-267f-4657-81cd-ab4996d4a813 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:06.218402 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb9b32-267f-4657-81cd-ab4996d4a813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:06.218667 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb9b32-267f-4657-81cd-ab4996d4a813 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:06.219349 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb9b32-267f-4657-81cd-ab4996d4a813 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:59:06.219765 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb9b32-267f-4657-81cd-ab4996d4a813 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:06.221426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb9b32-267f-4657-81cd-ab4996d4a813 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:06.234703 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1454/2901] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:59:06.240066 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdf33d-1cdc-4c7d-ae5e-c978a052a5ba None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:06.250111 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdf33d-1cdc-4c7d-ae5e-c978a052a5ba None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:06.255013 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376e734-36eb-4a89-a358-4c03effa8ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:06.255620 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376e734-36eb-4a89-a358-4c03effa8ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:06.255897 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376e734-36eb-4a89-a358-4c03effa8ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:06.256168 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376e734-36eb-4a89-a358-4c03effa8ec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:06.256812 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376e734-36eb-4a89-a358-4c03effa8ec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:06.258760 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376e734-36eb-4a89-a358-4c03effa8ec8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:06.358194 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1448/2902] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:59:06 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:59:06.359494 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb9b32-267f-4657-81cd-ab4996d4a813 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:06.371441 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb9b32-267f-4657-81cd-ab4996d4a813 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:06.433560 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311ad29-282a-4c30-8420-f874674826b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:06.434277 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311ad29-282a-4c30-8420-f874674826b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:06.434606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311ad29-282a-4c30-8420-f874674826b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:06.434802 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311ad29-282a-4c30-8420-f874674826b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:06.435464 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311ad29-282a-4c30-8420-f874674826b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:06.437575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311ad29-282a-4c30-8420-f874674826b2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:06.546604 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1455/2903] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:59:06 2022] POST /identity/v3/users => generated 282 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 08:59:06.552538 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376e734-36eb-4a89-a358-4c03effa8ec8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:06.563595 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376e734-36eb-4a89-a358-4c03effa8ec8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:06.569667 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9381cf9-c40b-4639-9402-bdcf689302a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:06.573489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9381cf9-c40b-4639-9402-bdcf689302a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:06.573489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9381cf9-c40b-4639-9402-bdcf689302a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:06.573489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9381cf9-c40b-4639-9402-bdcf689302a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:06.573489 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9381cf9-c40b-4639-9402-bdcf689302a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:06.574530 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9381cf9-c40b-4639-9402-bdcf689302a5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:06.585474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1449/2904] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:59:06.590281 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311ad29-282a-4c30-8420-f874674826b2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:06.602444 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311ad29-282a-4c30-8420-f874674826b2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:06.607418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40312a4e-9edd-4ba7-95c6-9798917f35b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:06.609513 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40312a4e-9edd-4ba7-95c6-9798917f35b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:06.609513 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40312a4e-9edd-4ba7-95c6-9798917f35b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:06.609513 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40312a4e-9edd-4ba7-95c6-9798917f35b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:06.609513 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40312a4e-9edd-4ba7-95c6-9798917f35b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:06.610960 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40312a4e-9edd-4ba7-95c6-9798917f35b2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:06.623370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1456/2905] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 08:59:06.627736 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9381cf9-c40b-4639-9402-bdcf689302a5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:06.638656 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9381cf9-c40b-4639-9402-bdcf689302a5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:06.643233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e9742-ef02-4cab-9d1f-656198601963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:06.643885 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e9742-ef02-4cab-9d1f-656198601963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:06.644159 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e9742-ef02-4cab-9d1f-656198601963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:06.644423 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e9742-ef02-4cab-9d1f-656198601963 admin admin] PATH_INFO: `/v3/projects/f9726462048f4ccebf247c373931f199/users/ad490a30dc75462b90531d85ca35d4d3/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:06.661305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e9742-ef02-4cab-9d1f-656198601963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9726462048f4ccebf247c373931f199, user_id=ad490a30dc75462b90531d85ca35d4d3, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:06.664518 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e9742-ef02-4cab-9d1f-656198601963 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:06.764824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1450/2906] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:59:06 2022] PUT /identity/v3/projects/f9726462048f4ccebf247c373931f199/users/ad490a30dc75462b90531d85ca35d4d3/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:59:06.775759 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2d54f-56d5-48e5-b541-ce968ba8f1f7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:06.776036 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2d54f-56d5-48e5-b541-ce968ba8f1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:06.776288 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2d54f-56d5-48e5-b541-ce968ba8f1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:06.805096 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a0d2d54f-56d5-48e5-b541-ce968ba8f1f7 None None] MFA Rules not processed for user `ad490a30dc75462b90531d85ca35d4d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:59:06.806352 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a0d2d54f-56d5-48e5-b541-ce968ba8f1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:06.824941 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:06.879520 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1457/2907] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:59:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:59:06.892049 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e9742-ef02-4cab-9d1f-656198601963 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:06.893932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a21e9742-ef02-4cab-9d1f-656198601963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:06.961048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e9742-ef02-4cab-9d1f-656198601963 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:06.964823 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcd3c2-ca8a-4d11-ba83-9177294b484f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:06.965499 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcd3c2-ca8a-4d11-ba83-9177294b484f service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:06.965774 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcd3c2-ca8a-4d11-ba83-9177294b484f service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:06.966041 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcd3c2-ca8a-4d11-ba83-9177294b484f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:06.975328 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcd3c2-ca8a-4d11-ba83-9177294b484f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:06.976906 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcd3c2-ca8a-4d11-ba83-9177294b484f service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:06.989298 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1451/2908] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:59:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 08:59:16.982888 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:17.136009 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=19 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:20.740030 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2d54f-56d5-48e5-b541-ce968ba8f1f7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:20.741525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a0d2d54f-56d5-48e5-b541-ce968ba8f1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:20.827426 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2d54f-56d5-48e5-b541-ce968ba8f1f7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:20.833027 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923b8d3-bea3-4ae1-8da8-d6ea95a7bd43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:20.833690 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923b8d3-bea3-4ae1-8da8-d6ea95a7bd43 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:20.833925 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923b8d3-bea3-4ae1-8da8-d6ea95a7bd43 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:20.834166 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923b8d3-bea3-4ae1-8da8-d6ea95a7bd43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:20.835986 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4923b8d3-bea3-4ae1-8da8-d6ea95a7bd43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:20.849254 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923b8d3-bea3-4ae1-8da8-d6ea95a7bd43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:20.850951 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923b8d3-bea3-4ae1-8da8-d6ea95a7bd43 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:20.864356 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1458/2909] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:59:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:59:25.977480 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcd3c2-ca8a-4d11-ba83-9177294b484f service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:25.979024 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c8dcd3c2-ca8a-4d11-ba83-9177294b484f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:25.997485 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcd3c2-ca8a-4d11-ba83-9177294b484f service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:26.013855 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c96df5-dcd7-413b-8315-0ca9f93b921a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:26.015523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c96df5-dcd7-413b-8315-0ca9f93b921a service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:26.015523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c96df5-dcd7-413b-8315-0ca9f93b921a service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:26.015523 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c96df5-dcd7-413b-8315-0ca9f93b921a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:26.017150 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-10c96df5-dcd7-413b-8315-0ca9f93b921a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:26.030388 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c96df5-dcd7-413b-8315-0ca9f93b921a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:26.032001 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c96df5-dcd7-413b-8315-0ca9f93b921a service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:26.044233 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1452/2910] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:59:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:59:30.858784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=34 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:36.039127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:46.697134 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923b8d3-bea3-4ae1-8da8-d6ea95a7bd43 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:46.760683 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923b8d3-bea3-4ae1-8da8-d6ea95a7bd43 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:46.765478 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:46.766107 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:46.766420 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:46.766720 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] PATH_INFO: `/v3/users/ad490a30dc75462b90531d85ca35d4d3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:46.767852 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad490a30dc75462b90531d85ca35d4d3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:46.769905 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:46.959186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad490a30dc75462b90531d85ca35d4d3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:46.959855 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad490a30dc75462b90531d85ca35d4d3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:47.043177 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad490a30dc75462b90531d85ca35d4d3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:47.051632 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad490a30dc75462b90531d85ca35d4d3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:47.063888 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad490a30dc75462b90531d85ca35d4d3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:47.069733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1459/2911] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:59:46 2022] DELETE /identity/v3/users/ad490a30dc75462b90531d85ca35d4d3 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:59:47.074333 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9564461-1489-4e96-8a61-0022ba98c24d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:47.074824 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9564461-1489-4e96-8a61-0022ba98c24d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:47.074949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9564461-1489-4e96-8a61-0022ba98c24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:47.093354 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d9564461-1489-4e96-8a61-0022ba98c24d None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:59:47.094574 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d9564461-1489-4e96-8a61-0022ba98c24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:47.155934 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1453/2912] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:59:47.171312 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:47.172285 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:47.233644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfae74-7764-4760-8725-5c533acb882d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:47.237589 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcce1e-1b78-443a-86f5-a0d37af73a89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:47.238492 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcce1e-1b78-443a-86f5-a0d37af73a89 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:47.238772 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcce1e-1b78-443a-86f5-a0d37af73a89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:47.239061 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcce1e-1b78-443a-86f5-a0d37af73a89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:47.248729 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcce1e-1b78-443a-86f5-a0d37af73a89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:47.250168 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcce1e-1b78-443a-86f5-a0d37af73a89 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:47.260085 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1460/2913] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:59:47.645441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9564461-1489-4e96-8a61-0022ba98c24d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:47.646871 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d9564461-1489-4e96-8a61-0022ba98c24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:47.659147 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9564461-1489-4e96-8a61-0022ba98c24d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:47.664351 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:47.665752 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:47.666065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:47.666519 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204 admin admin] PATH_INFO: `/v3/projects/f9726462048f4ccebf247c373931f199` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:47.668596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9726462048f4ccebf247c373931f199)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:47.670542 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:47.791277 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9726462048f4ccebf247c373931f199', 'request_id': 'req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:47.872533 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9726462048f4ccebf247c373931f199', 'request_id': 'req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:47.880229 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9726462048f4ccebf247c373931f199', 'request_id': 'req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:47.891526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204 admin admin] The token cache is being invalidate because project f9726462048f4ccebf247c373931f199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:59:47.892484 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:47.893761 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9374a69b-d4ea-45bf-b12f-6f1ab7d41204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:47.894706 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1454/2914] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:59:47 2022] DELETE /identity/v3/projects/f9726462048f4ccebf247c373931f199 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:59:47.899635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cfd45-41d9-40cc-a5f4-276e337405d1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:47.900111 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cfd45-41d9-40cc-a5f4-276e337405d1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:47.900223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cfd45-41d9-40cc-a5f4-276e337405d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:47.920143 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bb8cfd45-41d9-40cc-a5f4-276e337405d1 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:59:47.921256 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bb8cfd45-41d9-40cc-a5f4-276e337405d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:47.940710 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:47.983324 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1461/2915] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:59:47.991847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee573e3-09df-4415-b920-9d3630501942 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:47.999601 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee573e3-09df-4415-b920-9d3630501942 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:47.999601 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee573e3-09df-4415-b920-9d3630501942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:48.013041 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5ee573e3-09df-4415-b920-9d3630501942 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:59:48.014287 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5ee573e3-09df-4415-b920-9d3630501942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:48.022454 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1455/2916] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:59:48.026721 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cfd45-41d9-40cc-a5f4-276e337405d1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:48.036359 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cfd45-41d9-40cc-a5f4-276e337405d1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:48.040936 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007928b8-cdff-4468-85e9-4d87ad6be4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:48.041529 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007928b8-cdff-4468-85e9-4d87ad6be4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:48.041802 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007928b8-cdff-4468-85e9-4d87ad6be4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:48.042067 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007928b8-cdff-4468-85e9-4d87ad6be4b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:48.043022 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007928b8-cdff-4468-85e9-4d87ad6be4b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 08:59:48.043317 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007928b8-cdff-4468-85e9-4d87ad6be4b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:48.044899 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007928b8-cdff-4468-85e9-4d87ad6be4b0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:48.058187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1462/2917] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 08:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 08:59:48.062591 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee573e3-09df-4415-b920-9d3630501942 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:48.072773 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee573e3-09df-4415-b920-9d3630501942 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:48.077441 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e6d46-851b-49c7-8f29-863b2db00993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:48.078216 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e6d46-851b-49c7-8f29-863b2db00993 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:48.078342 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e6d46-851b-49c7-8f29-863b2db00993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:48.078578 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e6d46-851b-49c7-8f29-863b2db00993 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:48.079231 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e6d46-851b-49c7-8f29-863b2db00993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:48.081112 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e6d46-851b-49c7-8f29-863b2db00993 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:48.164400 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1456/2918] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 08:59:48 2022] POST /identity/v3/projects => generated 399 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:59:48.170456 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007928b8-cdff-4468-85e9-4d87ad6be4b0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:48.181614 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007928b8-cdff-4468-85e9-4d87ad6be4b0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:48.240750 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7b55d-e0f1-4766-8d54-9e8e25a5b021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:48.241375 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7b55d-e0f1-4766-8d54-9e8e25a5b021 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:48.241650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7b55d-e0f1-4766-8d54-9e8e25a5b021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:48.241919 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7b55d-e0f1-4766-8d54-9e8e25a5b021 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:48.242568 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7b55d-e0f1-4766-8d54-9e8e25a5b021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:48.244477 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7b55d-e0f1-4766-8d54-9e8e25a5b021 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:48.328861 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1463/2919] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 08:59:48 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 08:59:48.332985 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e6d46-851b-49c7-8f29-863b2db00993 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:48.343596 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e6d46-851b-49c7-8f29-863b2db00993 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:48.348944 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdba1b1-8136-4f2d-be8d-aabe4e30a07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:48.349669 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdba1b1-8136-4f2d-be8d-aabe4e30a07a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:48.349949 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdba1b1-8136-4f2d-be8d-aabe4e30a07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:48.350256 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdba1b1-8136-4f2d-be8d-aabe4e30a07a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:48.350898 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdba1b1-8136-4f2d-be8d-aabe4e30a07a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:48.352526 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdba1b1-8136-4f2d-be8d-aabe4e30a07a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:48.363780 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1457/2920] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:59:48.368074 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7b55d-e0f1-4766-8d54-9e8e25a5b021 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:48.378938 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7b55d-e0f1-4766-8d54-9e8e25a5b021 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:48.383543 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f4053-4a79-42ad-92cb-140aba7e2046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:48.384207 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f4053-4a79-42ad-92cb-140aba7e2046 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:48.384479 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f4053-4a79-42ad-92cb-140aba7e2046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:48.384743 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f4053-4a79-42ad-92cb-140aba7e2046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:48.385370 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f4053-4a79-42ad-92cb-140aba7e2046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:48.386989 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f4053-4a79-42ad-92cb-140aba7e2046 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:48.397742 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1464/2921] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 08:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 08:59:48.401895 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdba1b1-8136-4f2d-be8d-aabe4e30a07a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:48.412559 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdba1b1-8136-4f2d-be8d-aabe4e30a07a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:48.417104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec542a37-08a5-476a-9001-fac76d34447d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:48.417736 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec542a37-08a5-476a-9001-fac76d34447d admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:48.418008 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec542a37-08a5-476a-9001-fac76d34447d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:48.418305 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec542a37-08a5-476a-9001-fac76d34447d admin admin] PATH_INFO: `/v3/projects/80765c99af6c499c935b6468aef7d088/users/4907fcf9ed344886b170e53e4a709415/roles/dba75ce7d709483fbbb0027d29e99725` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:48.433325 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec542a37-08a5-476a-9001-fac76d34447d admin admin] RBAC: Authorizing `identity:create_grant(project_id=80765c99af6c499c935b6468aef7d088, user_id=4907fcf9ed344886b170e53e4a709415, role_id=dba75ce7d709483fbbb0027d29e99725)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:48.435922 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec542a37-08a5-476a-9001-fac76d34447d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:48.535374 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1458/2922] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 08:59:48 2022] PUT /identity/v3/projects/80765c99af6c499c935b6468aef7d088/users/4907fcf9ed344886b170e53e4a709415/roles/dba75ce7d709483fbbb0027d29e99725 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 08:59:48.550403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d127a8-cf29-4460-850b-c69306628352 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:48.550403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d127a8-cf29-4460-850b-c69306628352 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:48.550403 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d127a8-cf29-4460-850b-c69306628352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:48.577099 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e5d127a8-cf29-4460-850b-c69306628352 None None] MFA Rules not processed for user `4907fcf9ed344886b170e53e4a709415`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:59:48.578391 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e5d127a8-cf29-4460-850b-c69306628352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:48.652813 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1465/2923] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:59:48 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 08:59:48.669048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec542a37-08a5-476a-9001-fac76d34447d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:48.669048 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ec542a37-08a5-476a-9001-fac76d34447d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:48.693615 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:48.732705 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec542a37-08a5-476a-9001-fac76d34447d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:48.736317 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a340a-c618-4591-9e08-8f85c41c0aa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '434c9e87bda343659d2f410a6cdbf805', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:48.737474 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a340a-c618-4591-9e08-8f85c41c0aa3 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:48.737789 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a340a-c618-4591-9e08-8f85c41c0aa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:48.738065 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a340a-c618-4591-9e08-8f85c41c0aa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:48.746882 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a340a-c618-4591-9e08-8f85c41c0aa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:48.748303 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a340a-c618-4591-9e08-8f85c41c0aa3 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:48.762127 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1459/2924] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:59:48 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:59:57.811429 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d127a8-cf29-4460-850b-c69306628352 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:57.830807 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:57.876806 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d127a8-cf29-4460-850b-c69306628352 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:57.881330 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:57.881949 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:57.882253 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:57.882521 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] PATH_INFO: `/v3/users/4907fcf9ed344886b170e53e4a709415` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:57.883637 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4907fcf9ed344886b170e53e4a709415)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:57.885549 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:58.068682 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4907fcf9ed344886b170e53e4a709415'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:58.069349 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4907fcf9ed344886b170e53e4a709415'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:58.133848 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4907fcf9ed344886b170e53e4a709415'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:58.141463 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4907fcf9ed344886b170e53e4a709415'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:58.154338 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4907fcf9ed344886b170e53e4a709415'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:58.160231 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1466/2925] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 08:59:57 2022] DELETE /identity/v3/users/4907fcf9ed344886b170e53e4a709415 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 08:59:58.164691 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db245b-498e-44d2-8a3d-33087049c752 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:58.168477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db245b-498e-44d2-8a3d-33087049c752 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:58.168477 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db245b-498e-44d2-8a3d-33087049c752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:58.183838 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-00db245b-498e-44d2-8a3d-33087049c752 None None] MFA Rules not processed for user `e0cb001df8d24a87b7f9253f68745c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 08:59:58.185029 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-00db245b-498e-44d2-8a3d-33087049c752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:58.243161 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1460/2926] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 08:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 08:59:58.252767 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:58.254686 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:58.312852 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635421f-db4f-498b-a6c5-2a13a7fc8f55 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:58.317029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ed4ca-e2cc-46b4-ab30-bc5c28af952d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:58.317635 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ed4ca-e2cc-46b4-ab30-bc5c28af952d service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:58.317906 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ed4ca-e2cc-46b4-ab30-bc5c28af952d service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:58.318189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ed4ca-e2cc-46b4-ab30-bc5c28af952d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:58.329788 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ed4ca-e2cc-46b4-ab30-bc5c28af952d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:58.331187 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ed4ca-e2cc-46b4-ab30-bc5c28af952d service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:58.341578 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1467/2927] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:59:58.642429 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db245b-498e-44d2-8a3d-33087049c752 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:58.643841 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-00db245b-498e-44d2-8a3d-33087049c752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:58.721072 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db245b-498e-44d2-8a3d-33087049c752 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:58.726664 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bae66-1f63-4fd1-bd3d-2a6ffa59476a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d03db170fb6460794e32669456bcdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:58.727263 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bae66-1f63-4fd1-bd3d-2a6ffa59476a service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:58.727539 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bae66-1f63-4fd1-bd3d-2a6ffa59476a service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:58.727807 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bae66-1f63-4fd1-bd3d-2a6ffa59476a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:58.729520 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-105bae66-1f63-4fd1-bd3d-2a6ffa59476a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:58.742386 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105bae66-1f63-4fd1-bd3d-2a6ffa59476a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:58.743979 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105bae66-1f63-4fd1-bd3d-2a6ffa59476a service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:58.750042 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ed4ca-e2cc-46b4-ab30-bc5c28af952d service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 08:59:58.751291 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-411ed4ca-e2cc-46b4-ab30-bc5c28af952d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 08:59:58.756681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1461/2928] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 08:59:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 08:59:58.776886 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ed4ca-e2cc-46b4-ab30-bc5c28af952d service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 08:59:58.784606 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 08:59:58.785769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 08:59:58.786090 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 08:59:58.786428 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] PATH_INFO: `/v3/projects/80765c99af6c499c935b6468aef7d088` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 08:59:58.787460 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=80765c99af6c499c935b6468aef7d088)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 08:59:58.789419 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 08:59:58.891223 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 08:59:58.900629 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80765c99af6c499c935b6468aef7d088', 'request_id': 'req-9938d80c-113d-4bfc-a947-be92ecefbe0c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:58.979487 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80765c99af6c499c935b6468aef7d088', 'request_id': 'req-9938d80c-113d-4bfc-a947-be92ecefbe0c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:58.986073 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80765c99af6c499c935b6468aef7d088', 'request_id': 'req-9938d80c-113d-4bfc-a947-be92ecefbe0c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:58.998627 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] The token cache is being invalidate because project 80765c99af6c499c935b6468aef7d088 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 08:59:58.998924 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:58.999589 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 08:59:59.001163 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1468/2929] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 08:59:58 2022] DELETE /identity/v3/projects/80765c99af6c499c935b6468aef7d088 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 09:00:08.753698 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=76 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 09:00:09.168189 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 09:01:04.776730 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddfc42-1d54-4c11-a253-ab707047fc23 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 09:01:04.777377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddfc42-1d54-4c11-a253-ab707047fc23 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 09:01:04.777377 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddfc42-1d54-4c11-a253-ab707047fc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 09:01:04.796738 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-83ddfc42-1d54-4c11-a253-ab707047fc23 None None] MFA Rules not processed for user `ed50f5d8261d44b487390de092ef9136`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 09:01:04.797961 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-83ddfc42-1d54-4c11-a253-ab707047fc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:01:04.855802 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1462/2930] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 09:01:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 09:01:10.863943 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 09:01:10.865733 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:01:10.924662 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938d80c-113d-4bfc-a947-be92ecefbe0c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 09:01:10.928276 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a051b-4e52-4d2b-8115-ad7cb14efa3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 09:01:10.937784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a051b-4e52-4d2b-8115-ad7cb14efa3d service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 09:01:10.937784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a051b-4e52-4d2b-8115-ad7cb14efa3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 09:01:10.937784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a051b-4e52-4d2b-8115-ad7cb14efa3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 09:01:10.937784 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-901a051b-4e52-4d2b-8115-ad7cb14efa3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:01:10.943929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a051b-4e52-4d2b-8115-ad7cb14efa3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 09:01:10.945454 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a051b-4e52-4d2b-8115-ad7cb14efa3d service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 09:01:10.956138 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1469/2931] 10.222.0.44 () {60 vars in 1396 bytes} [Sun Sep 11 09:01:10 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 09:01:14.849108 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=25 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 09:01:20.952120 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=26 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 09:02:56.381172 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddfc42-1d54-4c11-a253-ab707047fc23 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 09:02:56.383605 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-83ddfc42-1d54-4c11-a253-ab707047fc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:02:56.472355 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddfc42-1d54-4c11-a253-ab707047fc23 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 09:02:56.477932 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184905f4-04a8-41f1-a55a-ca646edf5b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58c8502b91ea4ba79e85818727422410', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 09:02:56.478568 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184905f4-04a8-41f1-a55a-ca646edf5b11 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 09:02:56.478847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184905f4-04a8-41f1-a55a-ca646edf5b11 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 09:02:56.479117 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184905f4-04a8-41f1-a55a-ca646edf5b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 09:02:56.481847 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-184905f4-04a8-41f1-a55a-ca646edf5b11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:02:56.494820 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184905f4-04a8-41f1-a55a-ca646edf5b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 09:02:56.496487 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184905f4-04a8-41f1-a55a-ca646edf5b11 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 09:02:56.509931 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1463/2932] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 09:02:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 09:02:56.600841 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a051b-4e52-4d2b-8115-ad7cb14efa3d service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 09:02:56.609929 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a051b-4e52-4d2b-8115-ad7cb14efa3d service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 09:02:56.613584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a7cff-1f45-4e82-89f4-b738f7f77768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 09:02:56.614447 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7cff-1f45-4e82-89f4-b738f7f77768 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 09:02:56.614717 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7cff-1f45-4e82-89f4-b738f7f77768 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 09:02:56.615961 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a7cff-1f45-4e82-89f4-b738f7f77768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 09:02:56.619204 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3d6a7cff-1f45-4e82-89f4-b738f7f77768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:02:56.632438 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a7cff-1f45-4e82-89f4-b738f7f77768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 09:02:56.634391 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a7cff-1f45-4e82-89f4-b738f7f77768 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 09:02:56.643418 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1470/2933] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 09:02:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 09:02:56.797391 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184905f4-04a8-41f1-a55a-ca646edf5b11 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 09:02:56.810085 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184905f4-04a8-41f1-a55a-ca646edf5b11 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 09:02:56.813712 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57c29f-f761-4ccb-bcd0-0f906c7950a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 09:02:56.814341 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57c29f-f761-4ccb-bcd0-0f906c7950a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 09:02:56.814619 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57c29f-f761-4ccb-bcd0-0f906c7950a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 09:02:56.814894 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57c29f-f761-4ccb-bcd0-0f906c7950a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 09:02:56.816811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ff57c29f-f761-4ccb-bcd0-0f906c7950a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:02:56.831811 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57c29f-f761-4ccb-bcd0-0f906c7950a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 09:02:56.833245 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57c29f-f761-4ccb-bcd0-0f906c7950a9 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 09:02:56.845339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1464/2934] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 09:02:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 09:02:58.035849 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a7cff-1f45-4e82-89f4-b738f7f77768 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 09:02:58.037908 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3d6a7cff-1f45-4e82-89f4-b738f7f77768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:02:58.048298 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a7cff-1f45-4e82-89f4-b738f7f77768 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 09:02:58.051881 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850bcc2-db53-484c-9131-6ef47f902273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 09:02:58.053129 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850bcc2-db53-484c-9131-6ef47f902273 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 09:02:58.053417 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850bcc2-db53-484c-9131-6ef47f902273 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 09:02:58.053777 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850bcc2-db53-484c-9131-6ef47f902273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 09:02:58.055493 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2850bcc2-db53-484c-9131-6ef47f902273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:02:58.068985 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850bcc2-db53-484c-9131-6ef47f902273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 09:02:58.070489 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850bcc2-db53-484c-9131-6ef47f902273 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 09:02:58.082003 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1471/2935] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 09:02:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 09:03:06.839767 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=40 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 09:03:08.077551 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=12 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 09:03:11.962036 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57c29f-f761-4ccb-bcd0-0f906c7950a9 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 09:03:11.971104 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57c29f-f761-4ccb-bcd0-0f906c7950a9 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 09:03:11.974769 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a8aff-3a1e-4705-b76d-474eb0565663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92867e15dcf649adad78bf7a29a996f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 09:03:11.975865 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8aff-3a1e-4705-b76d-474eb0565663 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 09:03:11.976137 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8aff-3a1e-4705-b76d-474eb0565663 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 09:03:11.976433 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8aff-3a1e-4705-b76d-474eb0565663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 09:03:11.978164 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-271a8aff-3a1e-4705-b76d-474eb0565663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:03:12.039005 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a8aff-3a1e-4705-b76d-474eb0565663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 09:03:12.040421 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a8aff-3a1e-4705-b76d-474eb0565663 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 09:03:12.049777 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1465/2936] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 09:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 09:03:19.321448 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850bcc2-db53-484c-9131-6ef47f902273 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 09:03:19.322909 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2850bcc2-db53-484c-9131-6ef47f902273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:03:19.336665 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850bcc2-db53-484c-9131-6ef47f902273 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 09:03:19.341251 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 09:03:19.343575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 09:03:19.343575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 09:03:19.343575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] PATH_INFO: `/v3/users/ed50f5d8261d44b487390de092ef9136` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 09:03:19.343575 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed50f5d8261d44b487390de092ef9136)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 09:03:19.345404 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 09:03:19.537584 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed50f5d8261d44b487390de092ef9136'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:19.538644 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed50f5d8261d44b487390de092ef9136'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:19.617650 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed50f5d8261d44b487390de092ef9136'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:19.625190 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed50f5d8261d44b487390de092ef9136'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:19.637392 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed50f5d8261d44b487390de092ef9136'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:19.643361 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1472/2937] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 09:03:19 2022] DELETE /identity/v3/users/ed50f5d8261d44b487390de092ef9136 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 09:03:19.864339 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a8aff-3a1e-4705-b76d-474eb0565663 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 09:03:19.875123 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a8aff-3a1e-4705-b76d-474eb0565663 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 09:03:19.934319 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 09:03:19.934942 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 09:03:19.935218 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 09:03:19.935483 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] PATH_INFO: `/v3/projects/6d5504106ccb4549b09f29d7104d9b0a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 09:03:19.936609 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d5504106ccb4549b09f29d7104d9b0a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 09:03:19.938515 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 09:03:20.015958 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d5504106ccb4549b09f29d7104d9b0a', 'request_id': 'req-113eb80c-0da5-4c97-8701-b80c622dc164'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.068409 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d5504106ccb4549b09f29d7104d9b0a', 'request_id': 'req-113eb80c-0da5-4c97-8701-b80c622dc164'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.074125 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d5504106ccb4549b09f29d7104d9b0a', 'request_id': 'req-113eb80c-0da5-4c97-8701-b80c622dc164'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.085672 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] The token cache is being invalidate because project 6d5504106ccb4549b09f29d7104d9b0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 09:03:20.085957 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.086628 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.088109 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1466/2938] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 09:03:19 2022] DELETE /identity/v3/projects/6d5504106ccb4549b09f29d7104d9b0a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 09:03:20.092049 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 09:03:20.093416 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:03:20.106878 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3bec3-6b70-4e88-8490-26cb25a96695 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 09:03:20.160642 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 09:03:20.161252 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 09:03:20.161525 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 09:03:20.161790 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] PATH_INFO: `/v3/users/dc14bedd0ece41ab953ce98db4ab5abe` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 09:03:20.163555 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc14bedd0ece41ab953ce98db4ab5abe)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 09:03:20.165459 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 09:03:20.322979 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc14bedd0ece41ab953ce98db4ab5abe'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.324142 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc14bedd0ece41ab953ce98db4ab5abe'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.406186 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc14bedd0ece41ab953ce98db4ab5abe'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.413791 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc14bedd0ece41ab953ce98db4ab5abe'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.422029 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 09:03:20.426832 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc14bedd0ece41ab953ce98db4ab5abe'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.433178 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1473/2939] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 09:03:20 2022] DELETE /identity/v3/users/dc14bedd0ece41ab953ce98db4ab5abe => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 09:03:20.687727 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 09:03:20.689153 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:03:20.702970 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113eb80c-0da5-4c97-8701-b80c622dc164 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 09:03:20.731964 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 09:03:20.772187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455e4a9-0cd3-4f96-9485-10c08c939b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb001df8d24a87b7f9253f68745c35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '878028e7aaff4c28815b9cd862f1c36e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 09:03:20.772187 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455e4a9-0cd3-4f96-9485-10c08c939b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 09:03:20.772472 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455e4a9-0cd3-4f96-9485-10c08c939b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 09:03:20.772560 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455e4a9-0cd3-4f96-9485-10c08c939b87 admin admin] PATH_INFO: `/v3/projects/462948c022644502b584955878a2adbc` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 09:03:20.773681 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455e4a9-0cd3-4f96-9485-10c08c939b87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=462948c022644502b584955878a2adbc)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 09:03:20.775621 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455e4a9-0cd3-4f96-9485-10c08c939b87 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 09:03:20.883074 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2455e4a9-0cd3-4f96-9485-10c08c939b87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '462948c022644502b584955878a2adbc', 'request_id': 'req-2455e4a9-0cd3-4f96-9485-10c08c939b87'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.971543 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2455e4a9-0cd3-4f96-9485-10c08c939b87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '462948c022644502b584955878a2adbc', 'request_id': 'req-2455e4a9-0cd3-4f96-9485-10c08c939b87'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.977993 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2455e4a9-0cd3-4f96-9485-10c08c939b87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '462948c022644502b584955878a2adbc', 'request_id': 'req-2455e4a9-0cd3-4f96-9485-10c08c939b87'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.988991 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2455e4a9-0cd3-4f96-9485-10c08c939b87 admin admin] The token cache is being invalidate because project 462948c022644502b584955878a2adbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 09:03:20.989273 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2455e4a9-0cd3-4f96-9485-10c08c939b87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.989893 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2455e4a9-0cd3-4f96-9485-10c08c939b87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 09:03:20.992309 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1467/2940] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 09:03:20 2022] DELETE /identity/v3/projects/462948c022644502b584955878a2adbc => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 09:03:30.657263 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 09:03:31.160428 dv-cinder-856941-2-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 09:05:02.596821 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 09:05:02.598299 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:05:02.740844 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7a251-e3b0-443f-b892-62d48a99ca18 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 09:05:02.801889 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7aa192-1060-473f-9651-3ac7470ad1ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d03db170fb6460794e32669456bcdaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '811863e9c94d44a39647800f306505f0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 09:05:02.802526 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aa192-1060-473f-9651-3ac7470ad1ef service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 09:05:02.802801 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aa192-1060-473f-9651-3ac7470ad1ef service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 09:05:02.803070 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7aa192-1060-473f-9651-3ac7470ad1ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 09:05:02.804769 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4d7aa192-1060-473f-9651-3ac7470ad1ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 09:05:02.816818 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7aa192-1060-473f-9651-3ac7470ad1ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 09:05:02.818486 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7aa192-1060-473f-9651-3ac7470ad1ef service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 09:05:02.828329 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1474/2941] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 09:05:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 09:05:12.824092 dv-cinder-856941-2-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}